Compare commits

...
Author SHA1 Message Date
Carl Lerche a81e2722a4 chore: prepare v0.2.0 release (#1822) 2019-11-26 09:17:27 -08:00
Carl Lerche 4ddc437170 doc: add more doc_cfg annotations (#1821)
Also makes the `tokio::net::{tcp, udp, unix}` modules only for "utility"
types. The primary types are in `tokio::net` directly.
2019-11-25 14:32:55 -08:00
Carl Lerche 3ecaa6d91c docs: improve tokio::io API documentation (#1815)
Adds method level documentation for `tokio::io`.
2019-11-23 08:24:03 -08:00
leo-lb 0bc68adb34 tokio: remove performance regression notice (#1817) 2019-11-23 07:45:56 -08:00
Ivan Petkov e20dff39ce process: do not kill spawned processes on drop (#1814)
This updates the tokio `Command` and `Child` behavior to match that of
the stdlib: spawned processes will *not* be automatically killed when
the handle is dropped

Unlike the stdlib, any dropped (unix) processes may be reaped by tokio
behind-the-scenes after they exit and if new processes are awaited,
which mitigates the risks of piling up unreaped zombie unix processes

A `Command::kill_on_drop` method is added to allow the caller to
control whether the spawned child should be killed when the handle is
dropped. By default, this value is `false`.

The `Child::forget` method has been removed, as it is superseded by
`Command::kill_on_drop`
2019-11-22 20:10:05 -08:00
Carl Lerche 7b4c999341 default all feature flags to off (#1811)
Changes the set of `default` feature flags to `[]`. By default, only
core traits are included without specifying feature flags. This makes it
easier for users to pick the components they need.

For convenience, a `full` feature flag is included that includes all
components.

Tests are configured to require the `full` feature. Testing individual
feature flags will need to be moved to a separate crate.

Closes #1791
2019-11-22 15:55:10 -08:00
Carl Lerche e1b1e216c5 ci: bring back build tests (#1813)
This directory was deleted when `cargo hack` was introduced, however
there were some tests that were still useful (macro failure output).

Also, additional build tests will be added over time.
2019-11-22 14:38:49 -08:00
Taiki Endo 7cd63fb946 ci: use -Z avoid-dev-deps in features check instead of --no-dev-deps (#1812) 2019-11-22 14:13:18 -08:00
Carl Lerche bf741fec35 ci: generate docs (#1810)
Check docs as part of CI. This should catch link errors.
2019-11-22 11:55:57 -08:00
Carl Lerche 9b2aa14bb1 docs: annotate io mod with doc_cfg (#1808)
Annotates types in `tokio::io` module with their required feature flag.
This annotation is included in generated documentation.

Notes:

* The annotation must be on the type or function itself. Annotating just
  the re-export is not sufficient.

* The annotation must be **inside** the `pin_project!` macro or it is
  lost.
2019-11-22 09:56:08 -08:00
Carl Lerche 8546ff826d runtime: cleanup and add config options (#1807)
* runtime: cleanup and add config options

This patch finishes the cleanup as part of the transition to Tokio 0.2.
A number of changes were made to take advantage of having all Tokio
types in a single crate. Also, fixes using Tokio types from
`spawn_blocking`.

* Many threads, one resource driver

Previously, in the threaded scheduler, a resource driver (mio::Poll /
timer combo) was created per thread. This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from **outside** of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.

Now, instead of creating many resource drivers, a single resource driver
is used. Each scheduler thread will attempt to "lock" the resource
driver before parking on it. If the resource driver is already locked,
the thread uses a condition variable to park. Contention should remain
low as, under load, the scheduler avoids using the drivers.

* Add configuration options to enable I/O / time

New configuration options are added to `runtime::Builder` to allow
enabling I/O and time drivers on a runtime instance basis. This is
useful when wanting to create lightweight runtime instances to execute
compute only tasks.

* Bug fixes

The condition variable parker is updated to the same algorithm used in
`std`. This is motivated by some potential deadlock cases discovered by
`loom`.

The basic scheduler is fixed to fairly schedule tasks. `push_front` was
accidentally used instead of `push_back`.

I/O, time, and spawning now work from within `spawn_blocking` closures.

* Misc cleanup

The threaded scheduler is no longer generic over `P :Park`. Instead, it
is hard coded to a specific parker. Tests, including loom tests, are
updated to use `Runtime` directly. This provides greater coverage.

The `blocking` module is moved back into `runtime` as all usage is
within `runtime` itself.
2019-11-21 23:28:39 -08:00
Eliza Weisman 6866fe426c docs: expand and update crate-level docs (#1806)
## Motivation

Tokio's crate-level docs are currently pretty sparse, and in some cases
reference old names for APIs. Before 0.2 is released, they could use a
fresh coat of paint.

## Solution

This branch reworks and expands the `lib.rs` docs. In particular, I've
added a new "A Tour of Tokio" section, inspired by the [standard
library's similarly-named section][std]. This section lists all of
`tokio`'s public modules, and summarizes their major APIs. It also lists
the feature flags necessary to enable those APIs.

[std]: https://doc.rust-lang.org/std/index.html#a-tour-of-the-rust-standard-library

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-21 14:09:10 -08:00
Eliza Weisman d88846c4eb docs: update and expand the tokio::runtime API docs (#1804)
## Motivation

The `tokio::runtime` module's docs need to be updated to
track recent changes.

## Solution

This branch updates and expands the `runtime` docs.

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-20 17:46:35 -08:00
Eliza Weisman 7e6a10fccd docs: refresh tokio::io API docs (#1803)
## Motivation

The `tokio::io` module's docs are fairly sparse and not particularly up
to date. They ought to be improved before release.

## Solution

This branch adds new module-level docs to `tokio::io`. The new docs are
largely inspired by `std::io`'s documentation, and highlight the
similarities and differences between `tokio::io` and `std::io`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-20 15:09:38 -08:00
Carl Lerche 502cf5d95c io: flatten split module (#1802) 2019-11-20 14:45:38 -08:00
Eliza Weisman c223db3589 docs: improve tokio::task API documentation (#1801)
## Motivation

The new `tokio::task` module is pretty lacking in API docs. 

## Solution

This branch adds new API docs to the `task` module, including:

* Module-level docs with a summary of the differences between 
  tasks and threads
* Examples of how to use the `task` APIs in the module-level docs
* More docs for `yield_now`
* More docs and examples for `JoinHandle`, based on the 
  `std::thread::JoinHandle` API docs.

This branch contains commits cherry-picked from #1794 

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-20 14:36:45 -08:00
Carl Lerche 5cd665afd7 chore: update bytes dependency to git master (#1796)
Tokio will track changes to bytes until 0.5 is released.
2019-11-20 14:27:49 -08:00
Kevin Leimkuhler 3e643c7b81 time: Eagerly bind delays to timer (#1800)
## Motivation

Similar to #1666, it is no longer necessary to lazily register delays with the
executions default timer. All delays are expected to be created from within a
runtime, and should panic if not done so.

## Solution

`tokio::time` now assumes there to be a `CURRENT_TIMER` set when creating a
delay; this can be assumed if called within a tokio runtime. If there is no
current timer, the application will panic with a "no current timer" message.

## Follow-up

Similar to #1666, `HandlePriv` can probably be removed, but this mainly prepares
for 0.2 API changes. Because it is not in the public API, this can be done in a
following change.

Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-11-20 12:24:41 -08:00
Pen Tree bc150cd0b5 Fix doc links (#1799)
Link fix only. After this fix, `cargo doc --package` succeeds.
2019-11-20 12:24:17 -08:00
Carl Lerche 15dce2d11a net: flatten split mod (#1797)
The misc `split` types (`ReadHalf`, `WriteHalf`, `SendHalf`, `RecvHalf`)
are moved up a module and the `*::split` module is removed.
2019-11-20 11:29:32 -08:00
Taiki Endo d4fec2c5d6 chore: enable feature flag check on windows (#1798) 2019-11-20 07:05:50 -08:00
Carl Lerche 69975fb960 Refactor the I/O driver, extracting slab to tokio::util. (#1792)
The I/O driver is made private and moved to `tokio::io::driver`. `Registration` is
moved to `tokio::io::Registration` and `PollEvented` is moved to `tokio::io::PollEvented`.

Additionally, the concurrent slab used by the I/O driver is cleaned up and extracted to
`tokio::util::slab`, allowing it to eventually be used by other types.
2019-11-20 00:05:14 -08:00
Carl Lerche 7c8b8877d4 runtime: fix lost wakeup bug in scheduler (#1788)
When checking if a worker needs to be unparked, the SeqCst load does not
provide the necessary synchronization to ensure the scheduled task is
visible to the searching worker. The `load` is switched to
`fetch_add(0)` which does establish the necessary synchronization.

Adding unit tests catching this bug will require a fix to loom and will
be done at a later time. The bug fix has been validated with manual
testing.

Fixes #1768
2019-11-19 08:01:46 -08:00
Carl Lerche 0d38936b35 chore: refine feature flags (#1785)
Removes dependencies between Tokio feature flags. For example, `process`
should not depend on `sync` simply because it uses the `mpsc` channel.
Instead, feature flags represent **public** APIs that become available
with the feature enabled. When the feature is not enabled, the
functionality is removed. If another Tokio component requires the
functionality, it is stays as `pub(crate)`.

The threaded scheduler is now exposed under `rt-threaded`. This feature
flag only enables the threaded scheduler and does not include I/O,
networking, or time. Those features must be explictly enabled.

A `full` feature flag is added that enables all features.

`stdin`, `stdout`, `stderr` are exposed under `io-std`.

Macros are used to scope code by feature flag.
2019-11-18 07:00:55 -08:00
sclaire-1 13b6e9939e Edit CONTRIBUTING.md (#1784)
Edited the last sentence of the first section to improve clarity
2019-11-17 23:27:42 -08:00
Carl Lerche 44f10fe47f sync: require T: Clone for watch channels. (#1783)
There are limitations with `async/await` (no GAT) requiring the value to
be cloned on receive. The `poll` based API is not currently exposed.
This makes the `Clone` requirement explicit.
2019-11-17 09:03:44 -08:00
Carl Lerche c147be0437 make AtomicWaker private (#1782) 2019-11-16 23:35:17 -08:00
Carl Lerche b1d9e55487 task: move blocking fns into tokio::task (#1781) 2019-11-16 23:35:04 -08:00
Taiki Endo 66cbed3ce3 tls: enable test on CI (#1779) 2019-11-16 22:24:58 -08:00
Carl Lerche 4d19a99937 runtime: set spawn context on enter (#1780) 2019-11-16 22:24:28 -08:00
Taiki Endo 10dc659450 io: expose std{in, out, err} under io feature (#1759)
This exposes `std{in, out, err}` under io feature by moving
`fs::blocking` module into `io::blocking`.
As `fs` feature depends on `io-trait` feature, `fs` implementations can
always access `io` module.
2019-11-16 22:03:39 -08:00
Taiki Endo 320c84a433 chore: migrate from pin-project to pin-project-lite (#1778) 2019-11-16 09:14:40 -08:00
Carl Lerche 19f1fc36bd task: return JoinHandle from spawn (#1777)
`tokio::spawn` now returns a `JoinHandle` to obtain the result of the task:

Closes #887.
2019-11-16 08:28:34 -08:00
Carl Lerche 3f0eabe779 runtime: rename current_thread -> basic_scheduler (#1769)
It no longer supports executing !Send futures. The use case for
It is wanting a “light” runtime. There will be “local” task execution
using a different strategy coming later.

This patch also renames `thread_pool` -> `threaded_scheduler`, but
only in public APIs for now.
2019-11-16 07:19:45 -08:00
Taiki Endo 1474794055 runtime: allow non-unit type output in {Runtime, Spawner}::spawn (#1756) 2019-11-15 22:16:21 -08:00
Taiki Endo 92eb635669 net: add more impls for ToSocketAddrs (#1760) 2019-11-15 22:12:57 -08:00
Carl Lerche 8a7e57786a Limit futures dependency to Stream via feature flag (#1774)
In an effort to reach API stability, the `tokio` crate is shedding its
_public_ dependencies on crates that are either a) do not provide a
stable (1.0+) release with longevity guarantees or b) match the `tokio`
release cadence. Of course, implementing `std` traits fits the
requirements.

The on exception, for now, is the `Stream` trait found in `futures_core`.
It is expected that this trait will not change much and be moved into `std.
Since Tokio is not yet going reaching 1.0, I feel that it is acceptable to maintain
a dependency on this trait given how foundational it is.

Since the `Stream` implementation is optional, types that are logically
streams provide `async fn next_*` functions to obtain the next value.
Avoiding the `next()` name prevents fn conflicts with `StreamExt::next()`.

Additionally, some misc cleanup is also done:

- `tokio::io::io` -> `tokio::io::util`.
- `delay` -> `delay_until`.
- `Timeout::new` -> `timeout(...)`.
- `signal::ctrl_c()` returns a future instead of a stream.
- `{tcp,unix}::Incoming` is removed (due to lack of `Stream` trait).
- `time::Throttle` is removed (due to lack of `Stream` trait).
-  Fix: `mpsc::UnboundedSender::send(&self)` (no more conflict with `Sink` fns).
2019-11-15 22:11:13 -08:00
Markus Westerlind 930679587a codec: Remove Unpin requirement from Framed[Read,Write,] (#1758)
cc #1252
2019-11-15 16:30:07 +09:00
Carl Lerche 27e5b41067 reorganize modules (#1766)
This patch started as an effort to make `time::Timer` private. However, in an
effort to get the build compiling again, more and more changes were made. This
probably should have been broken up, but here we are. I will attempt to
summarize the changes here.

* Feature flags are reorganized to make clearer. `net-driver` becomes
  `io-driver`. `rt-current-thread` becomes `rt-core`.

* The `Runtime` can be created without any executor. This replaces `enter`. It
  also allows creating I/O / time drivers that are standalone.

* `tokio::timer` is renamed to `tokio::time`. This brings it in line with `std`.

* `tokio::timer::Timer` is renamed to `Driver` and made private.

* The `clock` module is removed. Instead, an `Instant` type is provided. This
  type defaults to calling `std::time::Instant`. A `test-util` feature flag can
  be used to enable hooking into time.

* The `blocking` module is moved to the top level and is cleaned up.

* The `task` module is moved to the top level.

* The thread-pool's in-place blocking implementation is cleaned up.

* `runtime::Spawner` is renamed to `runtime::Handle` and can be used to "enter"
  a runtime context.
2019-11-12 15:23:40 -08:00
Anton Barkovsky e3df2eafd3 tls: fix test certificate to work on macOS 10.15 (#1763)
macOS 10.15 introduced new requirements for certificates to be trusted:
https://support.apple.com/en-us/HT210176
2019-11-11 12:09:14 +01:00
Taiki Endo c15e01a09b chore: remove rust-toolchain and add minimum supported version check (#1748)
* remove rust-toolchain

* add minimum supported version check
2019-11-08 13:26:08 +09:00
Taiki Endo 64f2bf0072 chore: update CI config to test on stable (#1747) 2019-11-08 00:32:04 +09:00
Carl Lerche 7e35922a1d time: rename tokio::timer -> tokio::time (#1745) 2019-11-06 23:53:46 -08:00
Carl Lerche 4dbe6af0a1 runtime: misc pool cleanup (#1743)
- Remove builders for internal types
- Avoid duplicating the blocking pool when using the concurrent
  scheduler.
- misc smaller cleanup
2019-11-06 21:29:10 -08:00
leo-lb 9bec094150 timer: have example use delay_for instead of delay (#1735)
It is a more common use case that is to simply cause a delay for an amount of time.
I think it is more appropriate to show off `delay_for` in the example rather than `delay` that is useful only for less common use cases.
2019-11-06 21:28:21 -08:00
Taiki Endo 6f8b986bdb chore: update futures to 0.3.0 (#1741) 2019-11-07 05:09:10 +09:00
Carl Lerche 1a7f6fb201 simplify enter (#1736) 2019-11-06 09:51:15 -08:00
Carl Lerche 0da23aad77 fix clippy (#1737) 2019-11-05 23:38:52 -08:00
Carl Lerche d5c1119c88 runtime: combine executor and runtime mods (#1734)
Now, all types are under `runtime`. `executor::util` is moved to a top
level `util` module.
2019-11-05 19:12:30 -08:00
Carl Lerche a6253ed05a chore: unify all mocked loom files (#1732)
When the crates were merged, each component kept its own `loom` file
containing mocked types it needed. This patch unifies them all in one
location.
2019-11-04 22:22:40 -08:00
Carl Lerche 94f9b04b06 executor: switch some APIs to crate private. (#1731)
* switch `enter` to crate private
* make executor types pub(crate)
2019-11-04 14:12:24 -08:00
Carl Lerche 966ccd5d53 test: unify MockTask and task::spawn (#1728)
Delete `MockTask` in favor of `task::spawn`. Both are functionally
equivalent.
2019-11-03 14:10:14 -08:00
Taiki Endo 3948e16292 ci: install minimal profile by default (#1729) 2019-11-03 12:08:07 -08:00
Sebastian Dröge 6b35a1e8b0 impl AsyncWrite for std::io::Cursor (#1730)
Based on the implementation from the futures crate.
2019-11-03 21:21:01 +09:00
Carl Lerche e19bd77ef0 tests: fix bug + reorganize tests. (#1726)
Fixes a bug in the thread-pool executor related to shutdown
concurrent with a task that is self-notifying. A `loom` test is
added to validate the fix.

Additionally, in anticipation of the `thread_pool` module being
switched to private, tests are updated to use `Runtime` directly
instead of `thread_pool`. Those tests that cannot be updated
are switched to unit tests.
2019-11-02 17:03:06 -07:00
Carl Lerche c8fdbed27a chore: prune dev-dependencies
Most dev dependendencies are unused now that examples are moved into a
separate crate.
2019-11-02 09:40:37 +01:00
Carl Lerche 3e7d0be51d executor: remove Executor & TypedExecutor traits (#1724)
The `Executor` trait is sub-optimal as it forces a `Box<dyn Future>` to
spawn. Instead, `tokio::spawn` delegates to the specific runtime
implementation set for the current execution context.

`TypedExecutor`, while useful, has seen limited adoption. As such, it is
removed from `tokio` proper. Moving it to `tokio-util` is a possibility
that can be explored as follow up work.
2019-11-01 13:50:17 -07:00
Carl Lerche d70c928d88 runtime: merge multi & single threaded runtimes (#1716)
Simplify Tokio's runtime construct by combining both Runtime variants
into a single type. The execution style can be controlled by a
configuration setting on `Builder`.

The implication of this change is that there is no longer any way to
spawn `!Send` futures. This, however, is a temporary limitation. A
different strategy will be employed for supporting `!Send` futures.

Included in this patch is a rework of `task::JoinHandle` to support
using this type from both the thread-pool and current-thread executors.
2019-11-01 13:18:52 -07:00
Steven Fackler 742d89b0f3 Fix delay construction from non-lazy Handles (#1720)
Closes #1719.
2019-11-01 12:32:57 -07:00
Carl Lerche 20993341bd compat: extract crate to a dedicated git repo (#1723)
The compat crate is moved to https://github.com/tokio-rs/tokio-compat.
This allows pinning it to specific revisions of the Tokio git
repository. The master branch is intended to go through significant
churn and it will be easier to update the compat layer in batches.
2019-11-01 12:30:12 -07:00
Eliza Weisman e699d46534 compat: add a compat runtime (#1663)
## Motivation

The `futures` crate's [`compat` module][futures-compat] provides
interoperability between `futures` 0.1 and `std::future` _future types_
(e.g. implementing `std::future::Future` for a type that implements the
`futures` 0.1 `Future` trait). However, this on its own is insufficient
to run code written against `tokio` 0.1 on a `tokio` 0.2 runtime, if
that code also relies on `tokio`'s runtime services. If legacy tasks are
executed that rely on `tokio::timer`, perform IO using `tokio`'s
reactor, or call `tokio::spawn`, those API calls will fail unless there
is also a runtime compatibility layer.

## Solution

As proposed in #1549, this branch introduces a new `tokio-compat` crate,
with implementations of the thread pool and current-thread runtimes that
are capable of running both tokio 0.1 and tokio 0.2 tasks. The compat
runtime creates a background thread that runs a `tokio` 0.1 timer and
reactor, and sets itself as the `tokio` 0.1 executor as well as the
default 0.2 executor. This allows 0.1 futures that use 0.1 timer,
reactor, and executor APIs may run alongside `std::future` tasks on the
0.2 runtime.

### Examples

Spawning both `tokio` 0.1 and `tokio` 0.2 futures:

```rust
use futures_01::future::lazy;

tokio_compat::run(lazy(|| {
    // spawn a `futures` 0.1 future using the `spawn` function from the
    // `tokio` 0.1 crate:
    tokio_01::spawn(lazy(|| {
        println!("hello from tokio 0.1!");
        Ok(())
    }));

    // spawn an `async` block future on the same runtime using `tokio`
    // 0.2's `spawn`:
    tokio_02::spawn(async {
        println!("hello from tokio 0.2!");
    });

    Ok(())
}))
```

Futures on the compat runtime can use `timer` APIs from both 0.1 and 0.2
versions of `tokio`:

```rust
use std::time::{Duration, Instant};
use futures_01::future::lazy;
use tokio_compat::prelude::*;

tokio_compat::run_03(async {
    // Wait for a `tokio` 0.1 `Delay`...
    let when = Instant::now() + Duration::from_millis(10);
    tokio_01::timer::Delay::new(when)
        // convert the delay future into a `std::future` that we can `await`.
        .compat()
        .await
        .expect("tokio 0.1 timer should work!");
    println!("10 ms have elapsed");

    // Wait for a `tokio` 0.2 `Delay`...
    let when = Instant::now() + Duration::from_millis(20);
    tokio_02::timer::delay(when).await;
    println!("20 ms have elapsed");
});
```

## Future Work

This is just an initial implementation of a `tokio-compat` crate; there
are more compatibility layers we'll want to provide before that crate is
complete. For example, we should also provide compatibility between
`tokio` 0.2's `AsyncRead` and `AsyncWrite` traits and the `futures` 0.1
and `futures` 0.3 versions of those traits. In #1549, @carllerche also
suggests that the `compat` crate provide reimplementations of APIs that
were removed from `tokio` 0.2 proper, such as the `tcp::Incoming`
future.

Additionally, there is likely extra work required to get the 
`tokio-threadpool` 0.1 `blocking` APIs to work on the compat runtime.
This will be addressed in a follow-up PR.

Fixes: #1605
Fixes: #1552
Refs: #1549

[futures-compat]: https://rust-lang-nursery.github.io/futures-api-docs/0.3.0-alpha.19/futures/compat/index.html
2019-11-01 10:35:02 -07:00
Carl Lerche 72caede7be chore: remove dead files (#1718)
The `codec` module has been moved to `tokio-util`. Some files were left,
but they were never activated.
2019-11-01 21:30:06 +09:00
Carl Lerche 64c26ab1ee runtime: test creating a single-threaded runtime. (#1717) 2019-10-31 22:28:31 -07:00
Taiki Endo 02f7264008 chore: check each feature works properly (#1695)
It is hard to maintain features list manually, so use cargo-hack's
`--each-feature` flag. And cargo-hack provides a workaround for an issue
that dev-dependencies leaking into normal build (`--no-dev-deps` flag),
so removed own ci tool.

Also, compared to running tests on all features, there is not much
advantage in running tests on each feature, so only the default features
and all features are tested.
If the behavior changes depending on the feature, we need to test it as
another job in CI.
2019-10-31 21:09:32 -07:00
Jonathan Bastien-Filiatrault 2902e39db0 Allow non-destructive access to the read buffer. (#1600)
I need this to implement SMTP pipelining checks. I mostly need to
flush my send buffer when the read buffer is empty before waiting for
the next command.
2019-10-31 10:36:24 -04:00
Steven Fackler 630d3136dd timere: make Delay must_use (#1714)
Closes #1711
2019-10-30 20:21:03 -07:00
Sean McArthur 2c870b588f process: refactor OrphanQueue to use a Mutex instead fo SegQueue (#1712) 2019-10-30 15:29:04 -07:00
Jon Gjengset 109fd3086b thread-pool: in-place blocking with new scheduler (#1681)
The initial new scheduler PR omitted in-place blocking
support. This patch brings it back.
2019-10-30 08:58:49 -07:00
Sean McArthur e3261440e5 timer: inline CachePadded type (#1706) 2019-10-29 22:16:11 -07:00
Carl Lerche 2b909d6805 sync: move into tokio crate (#1705)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The sync implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-29 15:11:31 -07:00
Carl Lerche c62ef2d232 executor: move into tokio crate (#1702)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The executor implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-28 21:40:29 -07:00
Eliza Weisman 7eb264a0d0 net: replace RwLock<Slab> with a lock free slab (#1625)
## Motivation

The `tokio_net::driver` module currently stores the state associated
with scheduled IO resources in a `Slab` implementation from the `slab`
crate. Because inserting items into and removing items from `slab::Slab`
requires mutable access, the slab must be placed within a `RwLock`. This
has the potential to be a performance bottleneck especially in the context of
the work-stealing scheduler where tasks and the reactor are often located on
the same thread.

`tokio-net` currently reimplements the `ShardedRwLock` type from
`crossbeam` on top of `parking_lot`'s `RwLock` in an attempt to squeeze
as much performance as possible out of the read-write lock around the
slab. This introduces several dependencies that are not used elsewhere.

## Solution

This branch replaces the `RwLock<Slab>` with a lock-free sharded slab
implementation. 

The sharded slab is based on the concept of _free list sharding_
described by Leijen, Zorn, and de Moura in [_Mimalloc: Free List
Sharding in Action_][mimalloc], which describes the implementation of a
concurrent memory allocator. In this approach, the slab is sharded so
that each thread has its own thread-local list of slab _pages_. Objects
are always inserted into the local slab of the thread where the
insertion is performed. Therefore, the insert operation needs not be
synchronized.

However, since objects can be _removed_ from the slab by threads other
than the one on which they were inserted, removal operations can still
occur concurrently. Therefore, Leijen et al. introduce a concept of
_local_ and _global_ free lists. When an object is removed on the same
thread it was originally inserted on, it is placed on the local free
list; if it is removed on another thread, it goes on the global free
list for the heap of the thread from which it originated. To find a free
slot to insert into, the local free list is used first; if it is empty,
the entire global free list is popped onto the local free list. Since
the local free list is only ever accessed by the thread it belongs to,
it does not require synchronization at all, and because the global free
list is popped from infrequently, the cost of synchronization has a
reduced impact. A majority of insertions can occur without any
synchronization at all; and removals only require synchronization when
an object has left its parent thread.

The sharded slab was initially implemented in a separate crate (soon to
be released), vendored in-tree to decrease `tokio-net`'s dependencies.
Some code from the original implementation was removed or simplified,
since it is only necessary to support `tokio-net`'s use case, rather
than to provide a fully generic implementation.

[mimalloc]: https://www.microsoft.com/en-us/research/uploads/prod/2019/06/mimalloc-tr-v1.pdf

## Performance

These graphs were produced by out-of-tree `criterion` benchmarks of the
sharded slab implementation.


The first shows the results of a benchmark where an increasing number of
items are inserted and then removed into a slab concurrently by five
threads. It compares the performance of the sharded slab implementation
with a `RwLock<slab::Slab>`:

<img width="1124" alt="Screen Shot 2019-10-01 at 5 09 49 PM" src="https://user-images.githubusercontent.com/2796466/66078398-cd6c9f80-e516-11e9-9923-0ed6292e8498.png">

The second graph shows the results of a benchmark where an increasing
number of items are inserted and then removed by a _single_ thread. It
compares the performance of the sharded slab implementation with an
`RwLock<slab::Slab>` and a `mut slab::Slab`.

<img width="925" alt="Screen Shot 2019-10-01 at 5 13 45 PM" src="https://user-images.githubusercontent.com/2796466/66078469-f0974f00-e516-11e9-95b5-f65f0aa7e494.png">

Note that while the `mut slab::Slab` (i.e. no read-write lock) is
(unsurprisingly) faster than the sharded slab in the single-threaded
benchmark, the sharded slab outperforms the un-contended
`RwLock<slab::Slab>`. This case, where the lock is uncontended and only
accessed from a single thread, represents the best case for the current
use of `slab` in `tokio-net`, since the lock cannot be conditionally
removed in the single-threaded case.

These benchmarks demonstrate that, while the sharded approach introduces
a small constant-factor overhead, it offers significantly better
performance across concurrent accesses.

## Notes

This branch removes the following dependencies `tokio-net`:
- `parking_lot`
- `num_cpus`
- `crossbeam_util`
- `slab`

This branch adds the following dev-dependencies:
- `proptest`
- `loom`

Note that these dev dependencies were used to implement tests for the
sharded-slab crate out-of-tree, and were necessary in order to vendor
the existing tests. Alternatively, since the implementation is tested
externally, we _could_ remove these tests in order to avoid picking up
dev-dependencies. However, this means that we should try to ensure that
`tokio-net`'s vendored implementation doesn't diverge significantly from
upstream's, since it would be missing a majority of its tests.

Signed-off-by: Eliza Weisman <[email protected]>
2019-10-28 11:30:45 -07:00
Geoff Shannon 1195263584 Fix docs links: Redux (#1698) 2019-10-27 09:37:07 -07:00
Carl Lerche bccb713d98 thread-pool: test additional shutdown cases (#1697)
This adds an extra spawned task during the thread-pool shutdown loom
test. This results in additional cases being tested, primarily tasks
being stolen.
2019-10-26 22:15:39 -07:00
Linus Färnstrand 474befd23c chore: use argument position impl trait (#1690) 2019-10-26 08:40:38 -07:00
Carl Lerche 987ba7373c io: move into tokio crate (#1691)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `io` implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-26 08:02:49 -07:00
Carl Lerche 227533d456 net: move into tokio crate (#1683)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `net` implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-25 12:50:15 -07:00
Jon Gjengset 03a9378297 Make blocking pool non-static and use for thread pool (#1678)
Previously, support for `blocking` was done through a static `POOL` that
would spawn threads on demand. While this made the pool accessible at
all times, it made it hard to configure, and it was impossible to keep
multiple blocking pools.

This patch changes `blocking` to instead use a "default" global like the
ones used for timers, executors, and the like. There is now
`blocking::with_pool`, which is used by both thread-pool workers and the
current-thread runtime to ensure that a pool is available to tasks.

This patch also changes `ThreadPool` to spawn its worker threads on the
blocking pool rather than as free-standing threads. This is in
preparation for the coming in-place blocking work.

One downside of this change is that thread names are no longer
"semantic". All threads are named by the pool name, and individual
threads are not (currently) given names with numerical suffixes like
before.
2019-10-24 14:17:47 -07:00
Carl Lerche 99940aeeb4 chore: remove tracing. (#1680)
Historically, logging has been added haphazardly. Here, we entirely
remove logging as none of it is particularly useful. In the future, we
will add tracing back in order to expose useful data to the user of
Tokio.
2019-10-23 11:04:14 -07:00
Carl Lerche cfc15617a5 codec: move into tokio-util (#1675)
Related to #1318, Tokio APIs that are "less stable" are moved into a new
`tokio-util` crate. This crate will mirror `tokio` and provide
additional APIs that may require a greater rate of breaking changes.

As examples require `tokio-util`, they are moved into a separate
crate (`examples`). This has the added advantage of being able to avoid
example only dependencies in the `tokio` crate.
2019-10-22 10:13:49 -07:00
Carl Lerche b8cee1a60a timer: move tokio-timer into tokio crate (#1674)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `timer` implementation is now provided by the main `tokio` crate.
The `timer` functionality may still be excluded from the build by
skipping the `timer` feature flag.
2019-10-21 16:45:13 -07:00
Kevin Leimkuhler c9bcbe77b9 net: Eagerly bind resources to reactors (#1666)
## Motivation

The `tokio_net` resources can be created outside of a runtime due to how tokio
has been used with futures to date. For example, this allows a `TcpStream` to be
created, and later passed into a runtime:

```
let stream = TcpStream::connect(...).and_then(|socket| {
    // do something
});
tokio::run(stream);
```

In order to support this functionality, the reactor was lazily bound to the
resource on the first call to `poll_read_ready`/`poll_write_ready`. This
required a lot of additional complexity in the binding logic to support.

With the tokio 0.2 common case, this is no longer necessary and can be removed.
All resources are expected to be created from within a runtime, and should panic
if not done so.

Closes #1168

## Solution

The `tokio_net` crate now assumes there to be a `CURRENT_REACTOR` set on the
worker thread creating a resource; this can be assumed if called within a tokio
runtime. If there is no current reactor, the application will panic with a "no
current reactor" message.

With this assumption, all the unsafe and atomics have been removed from
`tokio_net::driver::Registration` as it is no longer needed.

There is no longer any reason to pass in handles to the family of `from_std` methods on `net` resources. `Handle::current` has therefore a more restricted private use where it is only used in `driver::Registration::new`.

Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-10-21 16:20:06 -07:00
Carl Lerche 978013a215 fs: move into tokio (#1672)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `fs` implementation is now provided by the main `tokio` crate. The
`fs` functionality may still be excluded from the build by skipping the
`fs` feature flag.
2019-10-21 15:49:00 -07:00
madmaxio 6aa6ebb5bc io: Take struct re-export to main crate (#1670) 2019-10-21 10:03:05 -07:00
Jonathas Conceição 4bee94eb06 runtime: update doc regarding runtime::run function helper (#1671) 2019-10-21 10:02:36 -07:00
Carl Lerche ed5a94eb2d executor: rewrite the work-stealing thread pool (#1657)
This patch is a ground up rewrite of the existing work-stealing thread
pool. The goal is to reduce overhead while simplifying code when
possible.

At a high level, the following architectural changes were made:

- The local run queues were switched for bounded circle buffer queues.
- Reduce cross-thread synchronization.
- Refactor task constructs to use a single allocation and always include
  a join handle (#887).
- Simplify logic around putting workers to sleep and waking them up.

**Local run queues**

Move away from crossbeam's implementation of the Chase-Lev deque. This
implementation included unnecessary overhead as it supported
capabilities that are not needed for the work-stealing thread pool.
Instead, a fixed size circle buffer is used for the local queue. When
the local queue is full, half of the tasks contained in it are moved to
the global run queue.

**Reduce cross-thread synchronization**

This is done via many small improvements. Primarily, an upper bound is
placed on the number of concurrent stealers. Limiting the number of
stealers results in lower contention. Secondly, the rate at which
workers are notified and woken up is throttled. This also reduces
contention by preventing many threads from racing to steal work.

**Refactor task structure**

Now that Tokio is able to target a rust version that supports
`std::alloc` as well as `std::task`, the pool is able to optimize how
the task structure is laid out. Now, a single allocation per task is
required and a join handle is always provided enabling the spawner to
retrieve the result of the task (#887).

**Simplifying logic**

When possible, complexity is reduced in the implementation. This is done
by using locks and other simpler constructs in cold paths. The set of
sleeping workers is now represented as a `Mutex<VecDeque<usize>>`.
Instead of optimizing access to this structure, we reduce the amount the
pool must access this structure.

Secondly, we have (temporarily) removed `threadpool::blocking`. This
capability will come back later, but the original implementation was way
more complicated than necessary.

**Results**

The thread pool benchmarks have improved significantly:

Old thread pool:

```
test chained_spawn ... bench:   2,019,796 ns/iter (+/- 302,168)
test ping_pong     ... bench:   1,279,948 ns/iter (+/- 154,365)
test spawn_many    ... bench:  10,283,608 ns/iter (+/- 1,284,275)
test yield_many    ... bench:  21,450,748 ns/iter (+/- 1,201,337)
```

New thread pool:

```
test chained_spawn ... bench:     147,943 ns/iter (+/- 6,673)
test ping_pong     ... bench:     537,744 ns/iter (+/- 20,928)
test spawn_many    ... bench:   7,454,898 ns/iter (+/- 283,449)
test yield_many    ... bench:  16,771,113 ns/iter (+/- 733,424)
```

Real-world benchmarks improve significantly as well. This is testing the hyper hello
world server using: `wrk -t1 -c50 -d10`:

Old scheduler:

```
Running 10s test @ http://127.0.0.1:3000
  1 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   371.53us   99.05us   1.97ms   60.53%
    Req/Sec   114.61k     8.45k  133.85k    67.00%
  1139307 requests in 10.00s, 95.61MB read
Requests/sec: 113923.19
Transfer/sec:      9.56MB
```

New scheduler:

```
Running 10s test @ http://127.0.0.1:3000
  1 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   275.05us   69.81us   1.09ms   73.57%
    Req/Sec   153.17k    10.68k  171.51k    71.00%
  1522671 requests in 10.00s, 127.79MB read
Requests/sec: 152258.70
Transfer/sec:     12.78MB
```
2019-10-19 11:09:40 -07:00
Steven Fackler 2a181320b7 fs: add read_to_string (#1664) 2019-10-16 15:47:37 -07:00
Taiki Endo 4c97e9dc28 fs: remove unnecessary trait and lifetime bounds (#1655) 2019-10-15 19:02:34 +09:00
Jon Gjengset 1cae04f8b3 macros: Use more consistent runtime names (#1628)
As discussed in #1620, the attribute names for `#[tokio::main]` and
`#[tokio::test]` aren't great. Specifically, they both use
`single_thread` and `multi_thread`, as opposed to names that match the
runtime names: `current_thread` and `threadpool`. This PR changes the
former to the latter.

Fixes #1627.
2019-10-12 12:55:39 -04:00
John-John Tedro 29f35df7f8 Remove incorrect FusedFuture impl on Delay (#1652)
`is_terminated` must return `true` until the future has been polled at least once to make sure that the associated block in select is called even after the delay has elapsed.

You use `Delay` in a `select!` by [fusing it](https://docs.rs/futures-preview/0.3.0-alpha.19/futures/future/trait.FutureExt.html#method.fuse):

```rust
let delay = tokio::timer::delay(/* ... */);
let delay = delay.fuse();

select! {
    _ = delay => {
        /* work here */
    }
}
```
2019-10-11 15:45:44 -04:00
Ivan Petkov 741bef8fe1 tokio: move signal and process reexports to crate root (#1643) 2019-10-11 11:00:39 -07:00
Carl Lerche 804dbd6f8e sync: fix mem leak in oneshot on task migration (#1648)
When polling the task, the current waker is saved to the oneshot state.
When the handle is migrated to a new task and polled again, the waker
must be swaped from the old waker to the new waker. In some cases, there
is a potential for the old waker to leak.

This bug was caught by loom with the recently added memory leak
detection.
2019-10-10 12:00:22 -07:00
Eliza Weisman 69fe65e972 io: add AsyncBufReadExt::split (#1642)
add a `split` method to `AsyncBufReadExt`, analogous to `std::io::BufRead::split`.
2019-10-09 13:17:07 -07:00
Jonathan Bastien-Filiatrault b8913ec7c0 executor: accurate idle thread tracking for the blocking pool (#1621)
Use a counter to count notifications. This protects against spurious
wakeups by pthreads and other libraries. The state transitions now
track num_idle precisely.
2019-10-07 14:04:28 -07:00
Eliza Weisman 8aa520e2bd io: add missing utility functions (#1632)
The standard library's `io` module has small utilities such as `repeat`,
`empty`, and `sink`, which return `Read` and `Write` implementations.
These can come in handy in some circiumstances. `tokio::io` has no
equivalents that implement `AsyncRead`/`AsyncWrite`.

This commit adds `repeat`, `empty`, and `sink` helpers to `tokio::io`.
2019-10-07 14:02:04 -07:00
Nick Stott ab2f71a612 chore: fix a comment typo (#1633) 2019-10-07 09:20:57 -07:00
Taiki Endo 42a5cb1508 timer: test arm on targets with target_has_atomic less than 64 (#1634) 2019-10-07 09:19:44 -07:00
Taiki Endo 2b4b0619d7 chore: update Cirrus CI config to test on beta (#1636) 2019-10-07 09:18:38 -07:00
Taiki Endo 55caddb9ce chore: do not trigger CI on std-future branch (#1635) 2019-10-07 09:17:27 -07:00
Vojtech Kral aefaef3abf tcp: export Incoming type (#1602) 2019-10-02 11:12:05 -07:00
Jon Gjengset c78c9168d7 macros: allow selecting runtime in tokio::test attr (#1620)
In the past, it was not possible to choose to use the multi-threaded
tokio `Runtime` in tests, which meant that any test that transitively
used `executor::threadpool::blocking` would fail with

```
'blocking' annotation used from outside the context of a thread pool
```

This patch adds a runtime annotation attribute to `#[tokio::test]` just
like `#[tokio::main]` has, which lets users opt in to the threadpool
runtime over `current_thread` (the default).
2019-10-02 10:58:34 -07:00
Jonathan Bastien-Filiatrault 9e1eef829a chore: annotate prelude re-exports as doc(no_inline) (#1601)
Fixes #1593 by making "use as _" linked in the documentation.
2019-10-02 10:55:35 -07:00
Taiki Endo f48980ae52 chore: update rust-toolchain to use beta (#1619) 2019-10-01 10:13:38 -04:00
Douman a1d1eb5eb3 macros: Allow arguments in non-main functions 2019-10-01 13:15:46 +02:00
Jon Gjengset 5efe31f2ed Prepare for release of 0.2.0-alpha.6 (#1617)
Note that `tokio-timer` and `tokio-tls` become 0.3.0-alpha.6 (not 0.2.0)
2019-09-30 18:35:52 -04:00
Jon Gjengset 5ce5a0a0e0 Fix for rust-lang/rust#64477 (#1618)
`foo(format!(...)).await` no longer compiles. There's a fix in
rust-lang/rust#64856, but this works around the problem.
2019-09-30 17:17:14 -04:00
Jon Gjengset 5fd5329497 Create BufStream from a BufReader + BufWriter (#1609)
This is handy if developers want to construct the inner buffers with a
particular capacity, and still end up with a `BufStream` at the end.
2019-09-30 14:22:59 -04:00
Taiki Endo 3b8ee2d991 chore: update futures-preview to 0.3.0-alpha.19 (#1610) 2019-09-30 13:32:37 -04:00
Jon Gjengset 7c341f45e0 chore: move CI to beta (#1615) 2019-09-27 09:51:45 -07:00
Jon Gjengset 611b4e11a7 Make Barrier::wait future Send (#1611)
It wasn't before. Now it is. And that is better.
2019-09-26 18:26:24 -04:00
Taiki Endo 159abb375f chore: update pin-project to 0.4 (#1603) 2019-09-27 04:51:28 +09:00
Carl Lerche 032b39487c sync: add spin_loop_hint to atomic waker (#1608)
The algorithm backing `AtomicWaker` effectively uses a spin lock backed
by notifying & yielding the current task. This adds a `spin_lock_hint`
annotation to cover this case.

While, in practice, the omission of `spin_lock_hint` would not cause
problems, there are platforms that do not handle spin locks very well
and could enter a deadlock in pathological cases.
2019-09-26 15:16:34 -04:00
Hung-I Wang b71b7b36be fs: update the doc comment of File::sync_data (#1596) 2019-09-25 09:05:50 -07:00
Taiki Endo c4567f741a io: add get_*/into_inner methods to BufStream (#1598) 2019-09-25 09:17:43 -04:00
Sean McArthur 18cef1901f tokio: add rt-current-thread optional feature
- Adds a minimum `rt-current-thread` optional feature that exports
  `tokio::runtime::current_thread`.
- Adds a `macros` optional feature to enable the `#[tokio::main]` and
  `#[tokio::test]` attributes.
- Adjusts `#[tokio::main]` macro to select a runtime "automatically" if
  a specific strategy isn't specified. Allows using the macro with only
  the rt-current-thread feature.
2019-09-24 12:17:04 -07:00
Taiki Endo c81447fdcc io: remove unsafe pin-projections and remove manual Unpin implementations (#1588)
* Removes most pin-projection related unsafe code.

* Removes manual Unpin implementations.
  As references always implement Unpin, there is no need to implement
  Unpin manually.

* Adds tests to check that Unpin requirement does not change accidentally 
  because changing Unpin requirements will be breaking changes.
2019-09-25 01:17:06 +09:00
Taiki Endo d50d050fae net: fix build-tests for uds (#1589) 2019-09-24 02:47:39 +09:00
Taiki Endo 3a55aba251 macros: add build tests for #[tokio::main] and #[tokio::test] (#1591) 2019-09-23 04:09:30 +09:00
Taiki Endo ddbb0c3836 macros: fix handling of arguments of #[tokio::main] attribute (#1578) 2019-09-23 03:05:04 +09:00
Taiki Endo 376d63867a chore: update pin-project to 0.4.0-beta.1 (#1586) 2019-09-23 01:52:14 +09:00
Taiki Endo eb2d0fbcd1 net: use Box::pin instead of Pin::new(Box::new) (#1587) 2019-09-22 09:28:55 -07:00
Jonathan Bastien-Filiatrault 695165feac timer: 32 bit ARM only has 32 bit atomics. (#1581) 2019-09-20 13:25:49 -07:00
Kirill Mironov ff186a4d03 tokio: add process feature (#1561) 2019-09-19 19:03:58 -07:00
Jon Gjengset 6611b32cce Export sync::Barrier from tokio::sync (#1577) 2019-09-19 21:32:35 -04:00
Carl Lerche 80ba2a4ff6 Release 0.2.0 alpha.5 (#1576) 2019-09-19 13:39:35 -07:00
Carl Lerche 8d09f61d33 net: fix build with only process (#1575) 2019-09-19 12:38:15 -07:00
Carl Lerche 815173f8e5 chore: rm tokio-buf (#1574)
The crate has not been updated and it does not seem like it is a good
path forward.
2019-09-19 12:11:21 -07:00
Markus Westerlind 34e388619f timer: delay_for should use tokio_timer::clock::now (#1572) 2019-09-19 11:20:18 -07:00
Jon Gjengset 9d5af20bcf Enable buffering both reads and writes (#1558)
`BufWriter` and `BufReader` did not previously forward the "opposite" trait (`AsyncRead` for `BufWriter` and `AsyncWrite` for `BufReader`). This meant that there was no way to have both directions buffered at once. This patch fixes that, and introduces a convenience type + constructor for this double-wrapped construct.
2019-09-19 14:17:15 -04:00
Jon Gjengset 613fde2637 sync: add Barrier primitive (#1571)
This adds `Barrier` to `tokio-sync`, which is an asynchronous alternative to [`std::sync::Barrier`](https://doc.rust-lang.org/std/sync/struct.Barrier.html). It is a synchronization primitive that allows multiple futures to "rendezvous" at certain points in their execution.
2019-09-19 14:16:56 -04:00
Jonathan Bastien-Filiatrault 22a3b10171 executor: fix blocking pool bug re: thread shutdown (#1562)
Currently, when threads in the blocking pool shutdown due to being idle
the counter tracking threads is not decremented. This prevents new threads
from being spawned to replace the shutdown threads.
2019-09-19 11:12:04 -07:00
Jon Gjengset e3415d8d61 sync: Make Lock more similar to std::sync::Mutex (#1573)
This renames `Lock` to `Mutex`, and brings the API more in line with `std::sync::Mutex`.

In partcular, locking now only takes `&self`, with the expectation that you place the `Mutex` in an `Arc` (or something similar) to share it between threads.

Fixes #1544.
Part of #1210.
2019-09-19 11:46:52 -04:00
Taiki Endo d1f60ac4c6 chore: deny warnings for doc tests (#1539) 2019-09-19 15:50:12 +09:00
Taiki Endo e2161502ad chore: fix clippy check failure (#1569) 2019-09-18 10:19:44 -07:00
yjh ab785bfba7 Update README.md (#1545)
change url's `version` to `latest`.
2019-09-17 10:54:22 -04:00
Lucio Franco 5f2f3f076d Add broken feature to old benchmarks (#1555)
Signed-off-by: Lucio Franco <[email protected]>
2019-09-13 14:04:23 -04:00
Taiki Endo efb27731ad timer: use our own AtomicU64 on targets with target_has_atomic less than 64 (#1538) 2019-09-13 10:18:32 -07:00
cynecx 578a9aec16 sync: replace deprecated mem::uninitialized usage with MaybeUninit (#1540) 2019-09-13 10:03:03 -07:00
Jonathan Bastien-Filiatrault 5b8fc19701 fs: propagate flush for stdout / stderr. (#1528) 2019-09-13 09:58:18 -07:00
Geoff Shannon c0a64d67ca chore: fix docs links (#1523) 2019-09-13 09:46:19 -07:00
Carl Lerche 6369d0f4f2 chore: add stability note to readme. (#1554) 2019-09-13 09:12:30 -07:00
Kirill Mironov f69ee652e6 tls: fix new temporary lifetime rustc error [E0597] (#1547)
Fixes: #1546
Signed-off-by: Kirill Mironov <[email protected]>
2019-09-11 10:22:04 -07:00
Jon Gjengset 9b3f8564af tls: Add get_ref and get_mut (#1537) 2019-09-04 17:45:53 -04:00
Ivan Petkov 9766cd644f process: omit several future types in favor of async/await (#1526) 2019-08-31 13:02:27 -07:00
Carl Lerche 431d4857e8 io: add Send / Sync impls for ReadHalf / WriteHalf (#1525) 2019-08-31 12:18:55 -07:00
Fenhl 26432355d5 tokio-process: Implement From<StdCommand> for Command (#1513) 2019-08-31 11:45:41 -07:00
Carl Lerche 2f91c85ad8 io: bring back split utility (#1521)
Bring back `split` utility as a free fn instead of a method on
`AsyncRead`. This utility wraps the `stream` in an `Arc` and uses mutual
exclusion to ensure correct access.

Additionally, the specialized `split_mut` fn on TcpStream and UdsStream
is promoted to `split`.
2019-08-30 20:46:07 -07:00
Fenhl 951827229a Add platform-specific methods to Command (#1516) 2019-08-30 18:28:41 -07:00
Geoff Shannon 383bb0a143 test: fix assert format args (#1520) 2019-08-30 14:03:37 -07:00
Benjamin Saunders d2bd6f5002 timer: Rename sleep to delay_for, reexport from tokio (#1518) 2019-08-30 10:23:54 -07:00
Carl Lerche 6a94d2cf4f tls: bump to v0.3.0-alpha.4 (#1515) 2019-08-30 10:20:44 -07:00
kellerkindt 4f99470d46 chore: fix compile error on latest nightly (#1512) 2019-08-30 09:15:05 -07:00
Jarred Nicholls 3d9134d13e executor: shut down idle threads in the blocking pool (#1514) 2019-08-30 08:26:16 -07:00
Sean McArthur 15dc0563b7 prepare v0.2.0-alpha.4 (#1509) 2019-08-29 12:59:10 -07:00
Sean McArthur 4e26258ac3 Re-add temporarily TcpStream::connect_std (#1508) 2019-08-29 11:45:17 -07:00
Carl Lerche a59e096c47 prepare v0.2.0-alpha.3 release (#1505) 2019-08-28 15:04:42 -07:00
Carl Lerche fc1640891e net: perform DNS lookup on connect / bind. (#1499)
A sealed `net::ToSocketAddrs` trait is added. This trait is not intended
to be used by users. Instead, it is an argument to `connect` and `bind`
functions.

The operating system's DNS lookup functionality is used. Blocking
operations are performed on a thread pool in order to avoid blocking the
runtime.
2019-08-28 13:25:50 -07:00
Jakub Beránek de9f05d4d3 docs: fix wording in tokio_process::Child documentation (#1502)
Fixes: #1494
2019-08-28 11:56:42 -04:00
Eliza Weisman 9c31797a08 net: switch from log to tracing (#1455)
* net: switch from `log` to `tracing`.

Motivation:

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime.

Solution:

This branch replaces the use of `log` in `tokio-net` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation.

Notes:

I removed the timing in `Reactor::poll` in favor of simply adding a
`#[tracing::instrument]` attribute. Since the generated `tracing` span
will have enter and exit events, a `tracing::Subscriber`
implemementation can use those to record timestamps, and process that
timing data in a much more sophisticated manner than including it in a
log line.

We can add the timestamps back if they're desired.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-27 17:53:57 -07:00
Ömer Sinan Ağacan d1c58b7940 tokio: export RunError in tokio::runtime::current_thread (#1487)
This type is used in return type of `Runtime::run`, but because the type
was not exported it was opaque in the documentation of `Runtime`.
2019-08-27 12:26:48 -07:00
Jacob Pratt 5f74a99ea3 implement spawn_with_handle in tokio_executor (#1492)
This code directly relies on `future-preview`'s `RemoteHandle`, and
exposes it via a `spawn_with_handle` method that is identical to
`future-preview`'s implementation.

Related: #1180
2019-08-27 12:26:11 -07:00
Carl Lerche 08e20fcf6a fs: add support for non-threadpool executors (#1495)
Provides a thread pool dedicated to running blocking operations (#588)
and update `tokio-fs` to use this pool.

In an effort to make incremental progress, this is an initial step
towards a final solution. First, it provides a very basic pool
implementation with the intend that the pool will be
replaced before the final release. Second, it updates `tokio-fs` to
always use this blocking pool instead of conditionally using
`threadpool::blocking`. Issue #588 contains additional discussion around
potential improvements to the "blocking for all" strategy.

The implementation provided here builds on work started in #954 and
continued in #1045. The general idea is th same as #1045, but the PR
improves on some of the details:

* The number of explicit operations tracked by `File` is reduced only to
  the ones that could interact. All other ops are spawned on the
  blocking pool without being tracked by the `File` instance.

* The `seek` implementation is not backed by a trait and `poll_seek`
  function. This avoids the question of how to model non-blocking seeks
  on top of a blocking file. In this patch, `seek` is represented as an
  `async fn`. If the associated future is dropped before the caller
  observes the return value, we make no effort to define the state in
  which the file ends up.
2019-08-27 12:25:20 -07:00
Carl Lerche 08099bb2d3 net: rewrite TcpStream::connect with async fn (#1497)
This also removes `TcpStream::connect_std` as the conversion functions
from `std` need to be rethought. A note tracking this has been added
to #1209.
2019-08-27 12:08:28 -07:00
Newton Ni 807d536846 codec: fix infinite loop in tokio_codec::LinesCodec (#1489) 2019-08-26 13:38:52 -07:00
Danny Browning 654f9d703f tokio: expose signal feature (#1491)
Expose tokio_net::signal::ctrl_c via tokio::net::signal::ctrl_c as feature signal.
2019-08-22 09:12:00 -07:00
Jon Gjengset a285689664 net: shutdown TCP write when asked to shut down (#1488) 2019-08-21 10:41:51 -07:00
Gurwinder Singh 13930eff2a chore: two async_await feature remained (#1486) 2019-08-21 18:53:42 +09:00
Taiki Endo 24fb33e012 io: add AsyncReadExt::{chain, take} (#1484) 2019-08-20 20:09:07 -07:00
Taiki Endo a791f4a758 chore: bump to newer nightly (#1485) 2019-08-20 20:07:16 -07:00
Eliza Weisman 7e7a5147a3 executor: switch from log to tracing (#1454)
## Motivation

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime. Updating the executor crate is an obvious
starting point. 

## Solution

This branch replaces the use of `log` in `tokio-executor` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation. I've also added a few `tracing` spans, primarily in
places where a variable is added to all the log messages in a scope.

## Notes

For users who are using the legacy `log` output, there is a feature flag
to enable `log` support in `tracing`. I thought about making this on by
default, but that would also enable the `tracing` dependency by default,
and it is only pulled in when the `threadpool` feature flag is enabled.
The `tokio` crate could enable the log feature in its default features
instead, since the threadpool feature is on by default in `tokio`. If
this isn't the right approach, I can change how `log` back-compatibility
is enabled.

We might want to consider adding more `tracing` spans in the threadpool
later. This could be useful for profiling, and for helping users debug
the way their applications interact with the executor. This branch is
just intended as a starting point so that we can begin emitting
`tracing` data from the executor; we should revisit what instrumentation
should be exposed, as well.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-20 12:44:26 -07:00
Jakub Beránek 2d56312b89 timer: introduce delay function shortcut (#1440)
This commit adds a simple delay shortcut to avoid writing Delay::new
everywhere and removes usages of Delay::new.
2019-08-20 08:39:55 -07:00
Ivan Petkov 357df38861 process: move into the tokio-net crate (#1475) 2019-08-19 19:42:54 -07:00
John-John Tedro 34a9dc2d76 Implement FusedStream and FusedFuture for Interval and Delay (#1476) 2019-08-19 10:21:34 -04:00
Ivan Petkov 68d5fcb8d1 docs: fix all rustdoc warnings (#1474) 2019-08-18 14:38:54 -07:00
Ivan Petkov 08b07afbd9 signal: remove new() constructors in favor of free functions (#1472)
* Also removed any `*_with_handle` related methods in favor of always
using the default reactor
2019-08-18 14:22:09 -07:00
Douman 7b0c60849c net: make default reactor guard public (#1468) 2019-08-18 11:36:21 -07:00
Ivan Petkov 6d8d388dc5 docs: add docs.rs metadata to build with all features (#1471) 2019-08-18 11:11:46 -07:00
Ivan Petkov bc61bd9d3d ci: ensure all tests are run for each feature (#1470)
* This includes running docs, examples, and lib tests for each added
feature, to ensure nothing is broken
2019-08-18 10:50:38 -07:00
Jakub Beránek a9585f0318 tokio-process: change CommandExt to a fully asynchronous Command struct (#1448)
Refs: #1371
2019-08-18 10:13:37 -07:00
Carl Lerche 88b4ec84d7 chore: prepare 0.2.0-alpha.2 release (#1465) 2019-08-17 23:34:25 -07:00
Philip Kannegaard Hayes 9f0daad5ac sync: fix fuzz_oneshot test by using instrumented loom::sync::Arc (#1464)
Since `tokio_sync::oneshot` makes a `CausalCell::with_mut()` mutable
access in the `Inner::drop()`, we must use the instrumented
`loom::sync::Arc`.

Uncovered by carllerche/loom#42
2019-08-17 21:30:31 -07:00
Carl Lerche c187cd75b6 signal: move into tokio-net (#1463) 2019-08-17 13:43:55 -07:00
Carl Lerche a83f5e4ba6 uds: move into tokio-net (#1462) 2019-08-16 14:42:05 -07:00
Carl Lerche 4935aae164 udp: remove files left over from moving tokio-udp (#1461) 2019-08-16 11:05:50 -07:00
Carl Lerche ba1829fd26 chore: rename ui-tests -> build-tests (#1460) 2019-08-16 09:26:56 -07:00
Carl Lerche ce7e60e396 udp: move tokio-udp into tokio-net (#1459) 2019-08-16 07:26:10 -07:00
Ivan Petkov d8b23ef852 signal: rename SignalKind methods (#1457)
This renames the SignalKind constructors to be a bit more readable
instead of using the signal names themselves
2019-08-15 21:09:09 -07:00
Carl Lerche 4788d3a9e3 tcp: move tokio-tcp into tokio-net (#1456) 2019-08-15 20:37:25 -07:00
Carl Lerche f1f61a3b15 net: reorganize crate in anticipation of #1264 (#1453)
Space is made to add `tcp`, `udp`, `uds`, ... modules.
2019-08-15 15:04:21 -07:00
Jakub Beránek d0a8e5d6f2 tokio-fs: rewrite std echo example using async/await (#1442)
This PR fixes the echo example in tokio-fs.

Refs: #1255
2019-08-15 13:10:17 -07:00
Carl Lerche 3b27dc31d2 threadpool: move threadpool into tokio-executor (#1452)
The threadpool is behind a feature flag.

Refs: #1264
2019-08-15 13:09:02 -07:00
Douman 37131b2114 runtime: refactor thread-local setters (#1449) 2019-08-15 13:00:57 -07:00
Carl Lerche 8538c25170 reactor: rename tokio-reactor -> tokio-net (#1450)
* reactor: rename tokio-reactor -> tokio-net

This is in preparation for #1264
2019-08-15 11:04:58 -07:00
Jakub Beránek 7b6438a172 tokio: rewrite print_each_packet example using async/await (#1446)
This PR fixes the print each packet example in tokio.

Refs: #1201
2019-08-15 10:42:34 -07:00
Carl Lerche 9de7083be8 executor: move current-thread into crate (#1447)
The `CurrentThread` executor is exposed using a feature flag.

Refs: #1264
2019-08-15 09:52:25 -07:00
John Doneth 8d55f98f6f udp: update tokio_udp::UdpFramed to std::future (#1370) 2019-08-14 11:18:21 -07:00
Taiki Endo 999a600494 io: add async BufReader/BufWriter (#1438) 2019-08-14 10:24:07 -07:00
Ilya Lakhin fb9809c068 executor, threadpool: forward port fix from #1155 (#1433)
Add executor::exit, allowing other executors inside threadpool::blocking.
2019-08-13 21:12:49 -07:00
Douman 517162792f macros: upgrade syn/quote (#1432) 2019-08-13 21:11:26 -07:00
Geoff Shannon fe90d61446 test: add a block_on function to tokio-test (#1431) 2019-08-13 21:10:26 -07:00
Ivan Petkov 338b37884a signal: Add SignalKind for registering signals more easily (#1430)
This avoids having consumers import libc for common signals, and it
improves discoverability since users need not be aware that libc
contains all supported constants.
2019-08-13 21:07:22 -07:00
Ivan Petkov 513326e01d signal: remove driver task for Windows event implementation (#1429)
Windows guarantees handler routines are always invoked in a new thread
(https://docs.microsoft.com/en-us/windows/console/handlerroutine), so we
don't need to use the handler-wake-another-driver technique used in the
Unix implementation

By broadcasting the event notifications from the handler, we no longer
need the Driver task to be spawned, which fixes the starvation issue if
the executor which runs the Driver task goes away

Also changed the behavior so that the default event handler runs if
all listeners for CTRL_{C, BREAK} events go away.
2019-08-13 21:01:06 -07:00
Ivan Petkov 73a91ad7b3 signal: delete blocking Read/Write impls on ChildStd{in, out, err} (#1428) 2019-08-13 20:53:02 -07:00
Taiki Endo 930cce8677 chore: update futures-preview to 0.3.0-alpha.18 (#1427) 2019-08-10 14:09:28 -07:00
Taiki Endo 6a125082e4 chore: apply unreachable_pub and missing_debug_implementations to all crates (#1424) 2019-08-11 04:28:52 +09:00
Taiki Endo d9f9c5658f chore: bump to newer nightly (#1426) 2019-08-11 02:01:20 +09:00
Taiki Endo fff39c03b1 ci: deny warnings in cirrus (#1425) 2019-08-11 01:41:51 +09:00
Tomasz Miąsko 756606a58b uds: implement split and split_mut for UnixStream (#1395)
This mirrors split API available in TcpStream.
2019-08-09 12:50:18 -07:00
Ran Benita e3b4c99a33 codec: a few suggestions (#1418)
How the buffer is managed is often critical for performance. Not
taking care of it will be catastrophic for performance beyond the
initial buffer size with the current implementation (a loop of
`reserve(1)`).
2019-08-09 12:20:31 -07:00
Taiki Endo 42fa0c28d3 timer: use std::sync::atomic::AtomicU64 instead of own AtomicU64 (#1421) 2019-08-10 03:42:03 +09:00
Taiki Endo f7b41c9dcc macros: improve error messages (#1420) 2019-08-09 10:28:22 -07:00
Taiki Endo 73102760cf chore: change default lint level to warning and deny warnings in CI (#1416) 2019-08-10 00:07:57 +09:00
Douman 18833a8e67 macros: Error on function with arguments (#1419) 2019-08-09 11:04:41 -04:00
tmiasko eba8bf2b4b io: implement AsyncWrite for Vec<u8> (#1409) 2019-08-08 20:55:27 -07:00
David Kellum 790d649dc5 update (dev dep) env_logger to latest 0.6 (#1390) 2019-08-08 20:37:32 -07:00
Lucio Franco 50e5d401df chore: prepare for v0.2.0-alpha.1 release (#1410) 2019-08-08 12:48:53 -07:00
Carl Lerche 2e69f2a7fd sync: track upstream loom changes (#1407) 2019-08-07 23:24:22 -07:00
Carl Lerche 962521f449 chore: enable full CI run (#1399)
* update all tests
* fix doc examples
* misc API tweaks
2019-08-07 20:02:13 -07:00
Carl Lerche 831be9c08e executor: remove unused dependency (#1406) 2019-08-07 19:55:42 -07:00
Carl Lerche 23c380a78f sync: track loom changes (#1405) 2019-08-07 15:38:34 -07:00
Lucio Franco 0a05332648 Remove git dep and add macro examples (#1404)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 15:02:38 -07:00
Lucio Franco 7268b0bb3a Migrate threadpool to futures-util (#1403)
* Migrate threadpool to futures-util

Signed-off-by: Lucio Franco <[email protected]>

* fmt
2019-08-07 16:26:44 -04:00
Lucio Franco 6412389bba executor: update park implementation (#1402)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 13:01:13 -07:00
tmiasko 53a94c025d io: implement AsyncBufRead for &[u8] and Cursor (#1397)
* `impl AsyncRead for &[u8]`
* `impl AsyncBufRead for &[u8]`
* `impl<T: AsRef<[u8]> + Unpin> AsyncRead for Cursor<T>`
* `impl<T: AsRef<[u8]> + Unpin> AsyncBufRead for Cursor<T>`
2019-08-07 12:57:37 -07:00
Gurwinder Singh 7174c63bf9 codec: move length delimited codec to tokio-codec (#1401) 2019-08-07 12:10:05 -07:00
Ivan Petkov cb2336ff3d process: Misc polish (#1400)
* Denied all warnings in tests, and denied rust_2018_idioms violations
* Bumped the crate version and set publish = false
* Pruned dependencies:
 - Only pull in tokio-sync on windows where it is used
 - Removed unused dev-dependencies
* Switch to Async{Read, Write} traits from tokio-io rather than
futures-io
* Use #[tokio::test] where possible
* Removed deprecated items
* Fix all doc examples
2019-08-07 10:38:45 -07:00
Carl Lerche 47e2ff48d9 tokio: fix API doc examples (#1396) 2019-08-06 14:03:49 -07:00
Carl Lerche 2f43b0a023 sync: polish and update API doc examples (#1398)
- Remove `poll_*` fns from some of the sync types.
- Move `AtomicWaker` and `Lock` to the root of the `sync` crate.
2019-08-06 13:54:56 -07:00
Carl Lerche 05d00aebb7 uds: remove poll_* fns in favor of async fns (#1394) 2019-08-05 15:05:02 -07:00
Carl Lerche 62733a6594 udp: remove poll_* fns in favor of async fns (#1393)
This removes the need for manual futures.
2019-08-05 14:18:18 -07:00
Carl Lerche 6d8cc4e475 tcp: update API documentation (#1392) 2019-08-05 11:50:55 -07:00
Carl Lerche 6cbe3d4f82 fs: use async fn instead of custom futures (#1381)
Also update all the doc examples.
2019-08-04 11:24:30 -07:00
Carl Lerche 337646b97f tokio: re-export future/stream utils (#1387) 2019-08-03 21:08:29 -07:00
Taiki Endo 0bb015588a codec: add AsyncBufRead/BufRead implementations (#1385)
* AsyncBufRead for FramedWrite2<T>
* BufRead for FramedWrite2<T>
* AsyncBufRead for Fuse<T, U>
* BufRead for Fuse<T, U>
2019-08-03 20:15:50 -07:00
Steven Fackler 63377e2110 Add AsyncWriteExt::shutdown (#1382) 2019-08-03 00:51:24 -04:00
Carl Lerche 878503f965 docs: update API documentation for some crates (#1380)
Updates API documentation for

- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
2019-08-02 14:35:32 -07:00
Carl Lerche 2c01b3e0e0 io: remove util from default features (#1379)
Sub-crates should require opting into features.
2019-08-02 12:59:24 -07:00
Carl Lerche ee9105d166 tokio: add async io traits to prelude (#1378) 2019-08-02 12:50:40 -07:00
Lucio Franco 5a4f849bba tokio: update tinyhttp example to async/await (#1372) 2019-08-02 12:24:15 -07:00
Lucio Franco ff41108834 io: move io helpers back into tokio-io (#1377)
Utilities are made optional with a feature flag.
2019-08-02 12:23:44 -07:00
Lucio Franco 6b202722ea io: Add AsyncWriteExt::flush (#1376)
* io: Add `AsyncWriteExt::flush`

* fmt

* fix clippy
2019-08-02 13:53:49 -04:00
Lucio Franco 144d980e5c tokio: update connect to async/await (#1375) 2019-08-02 10:03:06 -07:00
Lucio Franco 81d789b88f tokio: Update proxy to async/await (#1373) 2019-08-01 20:03:34 -07:00
Lucio Franco 634c19582f chore: add rust-toolchain file to track nightly version (#1374) 2019-08-01 20:00:55 -07:00
Ivan Petkov ff922bbe6d signal: Change constructors to return a result instead of lazy future (#1340) 2019-07-30 18:23:26 -07:00
Gurwinder Singh bf38631d6a chore: Fix spelling mistake (#1359) 2019-07-30 10:53:11 -07:00
Taiki Endo 6dda866191 tokio: re-enable StreamExt (#1362) 2019-07-30 09:55:34 -07:00
Taiki Endo 03e450deb1 sync: switch branch of loom dev-dependency to master (#1367)
* sync: switch branch of loom dev-dependency to master

* replace loom::fuzz with loom::model
2019-07-30 10:11:46 -04:00
andy finch fbf90e6356 Update process to use std::future (#1343) 2019-07-29 18:36:11 -07:00
Shell Chen 74168ae82f tcp: add async fn TcpStream::peek (#1360)
* tcp: add `async fn TcpStream::peek`

* tcp: apply rustfmt on tests
2019-07-26 10:55:02 -04:00
John Doneth d038009e7d Update chat example to async/await (#1349) 2019-07-25 19:44:23 -04:00
John Doneth 132e9f1da5 Update examples to return Result (#1305)
* update echo-udp

* update echo

* update hello_world

* update udp-client

* rustfmt

* remove send & sync

* rebase & change new updated examples
2019-07-25 16:47:31 -04:00
Taiki Endo fe021e6c00 ci: enable clippy lints (#1335) 2019-07-26 03:47:14 +09:00
Lucio Franco f311ac3d4f buf: Inital pass at updating BufStream (#1355) 2019-07-25 14:21:48 -04:00
Shell Chen 298be80249 tokio: include async-trait feature for uds (#1352) 2019-07-25 08:07:33 -07:00
John Doneth 79b017c773 Export LinesCodecError (#1350) 2019-07-24 15:26:41 -04:00
Taiki Endo ca0e5cc670 add TryFrom/From implementations (#1347)
* TryFrom<net::TcpListener> for TcpListener
* TryFrom<net::TcpStream> for TcpStream
* TryFrom<net::UdpSocket> for UdpSocket
* TryFrom<net::UnixDatagram> for UnixDatagram
* TryFrom<net::UnixListener> for UnixListener
* TryFrom<net::UnixStream> for UnixStream
* TryFrom<UnixDatagram> for mio_uds::UnixDatagram
* TryFrom<File> for io::File
* From<io::File> for File
2019-07-24 09:26:12 -07:00
Douman 59bc364a0e macros: detect double test attribute (#1336) 2019-07-22 09:28:07 -07:00
Taiki Endo e88d10a3cb chore: bump to newer nightly (#1338) 2019-07-22 06:04:02 +09:00
Ivan Petkov a3b8d82711 Merge tokio-process into tokio
Original repo can be found at https://github.com/alexcrichton/tokio-process/
2019-07-21 11:08:16 -07:00
Ivan Petkov d9688bc094 signal: change unix::Signal to return () instead of signum (#1330)
* This simplifies the API surface by returning () instead of the signal
number that was used during registration. This also more closely mirrors
the cross-platform `CtrlC` event stream API
* This is a **breaking change**
2019-07-20 15:12:53 -07:00
Ivan Petkov 320a5fdca7 signal: replace windows::Event with windows::CtrlBreak (#1331)
* Add a new `windows::CtrlBreak` struct which wil represent a stream of
CTRL_BREAK_EVENT signals on Windows systems
* The `windows::Event` type is no longer publicly accessible and is
replaced by using `CtrlC` or `windows::CtrlBreak`.

[breaking-change]
2019-07-20 10:50:27 -07:00
Taiki Endo 9af07ce208 chore: remove redundant field names in struct literals (#1334) 2019-07-20 10:43:19 -07:00
Taiki Endo 1b2d997863 chore: use ptr::{null, null_mut} instead of 0 as *{const, mut} (#1333) 2019-07-20 10:41:02 -07:00
Taiki Endo 7a52ddcd09 chore: remove unnecessary conversion (#1332) 2019-07-20 12:10:42 -04:00
Carl Lerche 9d3e5aac08 tokio: remove Send + 'static requirement from block_on (#1329)
Removes the `Send` requirement to futures passed to `Runtime::block_on`.
Previously, `block_on` was implemented by sending the future to a
runtime thread. In order to do this, the future must be Send.

The reason why the future is sent to the pool is because we cannot
guarantee, while off the pool, that a reactor / timer thread is running.
This is due to a limitation in the current version of tokio-threadpool.
There is a plan to fix this (#1177), but the proper fix is non trivial.

In order to unblock APIs that require this, this patch updates the
runtime to spawn an always running thread containing a reactor and
timer. All calls to `block_on` will use that reactor and timer.
2019-07-19 17:25:04 -07:00
Carl Lerche a99fa6e096 chore: remove tokio-futures facade crate (#1327)
This switches from using the tokio-futures facade to referencing
futures-* crates directly.
2019-07-19 13:11:46 -07:00
David Kellum b89ed00a0d Remove last non-dev dependency on rand crate (#1324)
Use std RandomState for XorShift seeding. This allows dropping _rand_
crate dep here, accept as a dev dependency for tests or benchmarks.
2019-07-19 12:12:32 -07:00
Dylan Frankland 12ce75f088 fs: add remove_dir_all and RemoveDirAllFuture (#1325)
Adds the sister function to `remove_dir` and mirrors the `create_dir_all` that's already exposed.
2019-07-19 12:09:53 -07:00
Taiki Endo a88308ed9f tokio: add AsyncReadExt::read_to_string (#1326) 2019-07-19 11:50:00 -07:00
João Oliveira a298472da8 tokio-tls: enable Send and Sync (#1317)
-  update 0 as *mut () calls to std::ptr::null_mut()
- impl Send and Sync for AllowStd
2019-07-19 10:21:26 -07:00
Shell Chen d0bb16192b timer: change Into to From trait for Elapsed (#1322) 2019-07-17 09:04:59 -04:00
Shell Chen a18ddb3b61 timer: impl Into<std::io::Error> for Elpased (#1321)
That convert Elpased to ErrorKind::TimedOut
2019-07-16 20:22:03 -07:00
Jon Gjengset 003b4d8074 Get rid of Enter for with_default (#1315)
We want executors to enforce that there are never multiple active at the
same time. This is ensured through `Enter`, which will panic if you
attempt to create more than one. However, by requiring you to pass an
`&mut Enter` to `executor::with_default`, we were *also* disallowing
temporarily overriding the current executor.

This patch removes that requirement.
2019-07-16 14:29:35 -04:00
Yin Guanhao 6d186fe40e Replace (some) uninitialized with MaybeUninit (#1295) 2019-07-16 10:47:46 -07:00
Diggory Blake 0d99ddd4f4 tcp: implement "split_mut" for TcpStream (#1289) 2019-07-16 10:28:00 -07:00
João Oliveira 448d9d2eab tls: update to std-future (#1224) 2019-07-16 10:26:08 -07:00
David Kellum 0de3a69eb4 fs: drop deprecated tempdir crate use in tests (#1312)
In particular because it pulls in old rand duplicates. Replace use
with tempfile::tempdir() which has been available since tempfile
3.0.0.
2019-07-15 15:32:33 -07:00
John Doneth 61aee5fc28 examples: pdate tinydb example (#1288)
Update tinydb example to use async / await.
2019-07-15 15:19:36 -07:00
Sean McArthur 7f7f74985e io: Minor adjustments to tokio-test IO (#1306)
This also re-exports `bytes::{Buf, BufMut}` from `tokio-io`.
2019-07-15 14:53:16 -07:00
Jon Gjengset e6cf976662 tokio: include async-traits feature (#1314)
The `tokio` facade crate will depend on the `async-traits` feature flag in
sub crates.
2019-07-15 14:02:14 -07:00
Taiki Endo b14e189e44 add #[must_use] to more futures and streams (#1309) 2019-07-15 13:28:56 -07:00
Taiki Endo 2dde2b448f Fix import of ready macro 2019-07-15 11:52:13 -07:00
Taiki Endo 6742816e78 tokio: add AsyncBufReadExt::lines 2019-07-15 11:52:13 -07:00
Taiki Endo ab040bb498 tokio: add AsyncBufReadExt::read_line 2019-07-15 11:52:13 -07:00
Taiki Endo 0cfa120ba8 tokio: add AsyncBufReadExt::read_until 2019-07-15 11:52:13 -07:00
Taiki Endo 5774a9cd64 io: add AsyncBufRead trait 2019-07-15 11:52:13 -07:00
John Doneth da49ede41e update udp-codec example (#1293) 2019-07-15 14:14:03 -04:00
Carl Lerche d224d6415e chore: indicate the master branch docs are old. (#1304)
Fixes #1292
2019-07-15 10:44:47 -07:00
Gurwinder Singh 83273b8b50 chore: use ready macro from futures-core (#1300) 2019-07-15 10:43:54 -07:00
Taiki Endo ca708d6d87 chore: update rand dependency to 0.7 (#1302) 2019-07-15 10:13:10 -07:00
matthieugras 0b75c0c53d executor: block thread when needed in block fn (#1303)
Fix #1296
2019-07-15 08:56:20 -07:00
Alex Gaynor 5fbb36a060 reactor: bump parking_lot dependency (#1298) 2019-07-14 09:28:54 -07:00
Gurwinder Singh c897a5b696 Re-export tokio-fs (#1287) 2019-07-14 12:03:49 -04:00
Sean McArthur 48d7f7b931 tokio-test: add tokio_test::io mock builder 2019-07-12 11:19:12 -07:00
Carl Lerche 2291823181 tokio: re-export correct tokio-uds version (#1286)
An earlier PR (#1282) re-exported the version from crates.io and not git
master.
2019-07-11 10:27:51 -07:00
andy finch 795e02f4c6 fs: update to use std::future (#1269) 2019-07-11 09:05:49 -07:00
Carl Lerche 7ac8bfc821 chore: bump to newer nightly (#1284) 2019-07-10 14:36:36 -07:00
Carl Lerche a79483750f tokio: update echo example (#1283) 2019-07-10 14:21:20 -07:00
Carl Lerche 3855f373d3 tokio: re-export tokio-uds (#1282)
The tokio-uds crate has been previously updated to std::future. This
commit enables the re-export in the tokio facade crate.
2019-07-10 11:21:27 -07:00
Carl Lerche bd3f3270db tokio: update threaded runtime to std::future (#1280)
re-enables the threaded runtime and sets it (again) as the default.
2019-07-10 11:21:06 -07:00
Taiki Endo e5525628cd chore: remove usage of deprecated ONCE_INIT (#1281) 2019-07-10 08:35:07 -07:00
Carl Lerche f1b8a318d9 tokio: add AsyncReadExt::read_to_end (#1279) 2019-07-09 16:17:58 -07:00
Carl Lerche 64343f1b78 tokio: add AsyncWriteExt::write_all (#1277) 2019-07-09 12:37:14 -07:00
Ruben De Smet 82795184c1 tokio: rewrite examples with async. (#1228) 2019-07-09 11:21:12 -07:00
Thomas Lacroix f529928d87 chore: script updating versions in links to docs.rs (#1249) 2019-07-09 11:19:38 -07:00
Ivan Petkov 461eebe612 signal: Replace ctrl_c with a CtrlC struct (#1273)
* Add a new `CtrlC` struct which will represent a stream of SIGINT
signals on Unix or the CTRL_C event on Windows
* `CtrlC` implements `Stream<Output = ()>` rather than `IoSteam` as
previously
2019-07-09 08:48:46 -07:00
Gurwinder Singh 407d15cf93 chore: Add link to docs (#1276) 2019-07-09 11:21:29 -04:00
Yin Guanhao 80915906d8 uds: update to std-future (#1227) 2019-07-08 14:58:40 -07:00
Yin Guanhao 88e775dcf0 udp: UdpSocket split support (#1226) 2019-07-08 14:47:31 -07:00
Carl Lerche 8b49a1e05f chore: update examples link in README (#1274) 2019-07-08 13:34:39 -07:00
Carl Lerche 8fa1510d67 timer: fix build (#1275) 2019-07-08 11:23:15 -07:00
Reto Kaiser 7797a377c3 current-thread: make tokio_current_thread::Handle Sync (#1119) 2019-07-08 10:25:34 -07:00
Steven Fackler b62d224fac timer: fix Handle::timeout (#1093)
The old implementation didn't work for Timeout<Stream>, since the method
took a deadline rather than a timeout.
2019-07-08 10:18:37 -07:00
Aaron Hill d4803bc868 Use Sink trait from futures-sink-preview (#1244) 2019-07-08 09:56:11 -07:00
Thomas Lacroix e07a03b3c5 signal: update instructions in Ctrl-C example (#1270)
Fixes: #1248
2019-07-07 09:49:19 -07:00
Taiki Endo 7b86acb71d chore: Update futures-preview to 0.3.0-alpha.17 (#1267) 2019-07-04 14:34:57 -07:00
Steffen Butzer 0651f09427 Remove usage of deprecated std::error::Error methods (#1206) (#1245) 2019-07-03 23:06:03 -07:00
Thomas Lacroix 516251052d Add missing links in README.md (#1233)
Fixes: #1229
2019-07-03 22:59:10 -07:00
Ivan Petkov cbad83f362 signal: migrate to std::futures (#1218)
Migrate to std::futures and the futures 0.3 preview and use async/await
where possible

**Breaking change:** the IoFuture and IoStream definitions used to refer
to Box<dyn Future> and Box<dyn Stream>, but now they are defined as
Pin<...> versions which are technically breaking.

No other breaking or functional changes have been made
2019-07-03 10:40:59 -07:00
Eliza Weisman bd9760e124 add release documentation to CONTRIBUTING.md (#1171)
## Motivation

Currently, the process for releasing a new version of a Tokio crate is
somewhat complex, and is not well-documented. To make it easier for
contributors to release minor versions more frequently, there should be
documentation describing this process.

## Solution

This branch adds a section to `CONTRIBUTING.md` describing how to
release a new version of a Tokio crate. The steps are based on those
described by @carllerche in an offline conversation.

I've also added a quick shell script to actually publish new crate 
versions. This should make it harder to make mistakes when 
publishing.

Signed-off-by: Eliza Weisman <[email protected]>
2019-07-03 10:18:02 -07:00
Carl Lerche 3e898f58a5 tcp: add ascyc fn TcpListener::accept (#1242)
Refs: #1209
2019-07-03 09:49:56 -07:00
Ivan Petkov c531865d2c ci: don't generate docs for deps on FreeBSD (#1241) 2019-07-03 09:41:35 -07:00
Ivan Petkov 722eb257be ci: scope each tests/examples invocation to a specific crate (#1238) 2019-07-03 08:49:05 -07:00
Taiki Endo ceed29586b io: fix documents (#1231) 2019-07-01 20:44:12 -07:00
Carl Lerche 70eca184f0 tokio: re-enable timer in runtimes (#1237)
This also brings back the timer tests in the tokio crate.
2019-07-01 18:27:13 -07:00
Carl Lerche b2c777846e timer: finish updating timer (#1222)
* timer: restructure feature flags
* update timer tests
* Add `async-traits` to CI

This also disables a buggy `threadpool` test. This test should be fixed in the future.

Refs #1225
2019-06-30 08:48:53 -07:00
Lucio Franco 8e7d8af588 docs: add note in the readme about the master branch (#1230) 2019-06-29 21:47:20 -04:00
Yin Guanhao 7380dd2482 TcpSocket specialized split (#1217) 2019-06-28 23:36:49 -07:00
Eliza Weisman af46eac583 chore: remove tokio-trace, add "Related Projects" to README (#1221)
## Motivation

The `tokio-trace` and `tokio-trace-core` crates have been renamed to
`tracing` and `tracing-core`, and moved to their own repository
(`tokio-rs/tracing`).

## Solution

This branch removes `tokio-trace` and `tokio-trace-core` from the
`tokio` repository. In addition, I've added a "Related Projects" section
to the root README, which lists `tracing` (as well as  `mio`, and
`bytes`) as other libraries maintained by the Tokio project. I thought
that this would help folks looking for `tokio-trace` here find it in its
new home.

In addition, it changes `tokio` to depend on `tracing-core` rather than
`tokio-trace-core`.

Closes #1159

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-28 13:13:46 -07:00
Carl Lerche e7488d983e threadpool: update to std::future (#1219)
An initial pass at updating `tokio-threadpool` to `std::future`. The
codebase and tests both now run using `std::future` but the wake
mechanism is not ideal. Follow up work will be required to improve on
this.

Refs: #1200
2019-06-27 22:30:56 -07:00
Sean McArthur e4415d986a sync: change oneshot poll_close to poll_closed
The action of `Sender::poll_close` is to check if the receiver has been
closed, not to try to close the sender itself. So change to
`poll_closed`.
2019-06-27 13:56:58 -07:00
Carl Lerche ff906acdfb ci: disable cache on cirrus (#1215)
Caching takes longer than rebuilding
2019-06-27 12:08:43 -07:00
Carl Lerche 32ceccb465 sync: add async APIs to oneshot and mpsc (#1211)
Adds:

- oneshot::Sender::close
- mpsc::Receiver::recv
- mpsc::Sender::send

Also renames `poll_next` to `poll_recv`.

Refs: #1210
2019-06-27 11:33:36 -07:00
Douman 0af05e7408 macros: allow configuring runtime used by main macro (#1185) 2019-06-27 10:40:21 -07:00
jesskfullwood 6b9e7bdace codec: update to use std-future (#1214)
Strategy was to

- copy the old codec code that was temporarily being stashed in `tokio-io`
- modify all the type signatures to use Pin, as literal a translation as possible
- fix up the tests likewise

This is intended just to get things compiling and passing tests. Beyond that there is surely
lots of refactoring that can be done to make things more idiomatic. The docs are unchanged.

Closes #1189
2019-06-27 10:10:29 -07:00
Carl Lerche ed4d4a5353 chore: format code and enable rustfmt CI task (#1212) 2019-06-27 00:05:01 -07:00
Carl Lerche 1f47ed3dcc tokio: rewrite io_read.rs test to use async/await (#1207)
This simplifies the test
2019-06-26 17:06:56 -07:00
Carl Lerche e9aaacddbd tokio: re-export sync::{lock,mpsc} (#1208)
These types have been updated already.
2019-06-26 16:54:15 -07:00
Carl Lerche 11f6b2862f tokio: move I/O helpers to ext traits (#1204)
Refs: #1203
2019-06-26 14:42:19 -07:00
Carl Lerche 8404f796ac test: get cargo test --tests working (#1205)
Broken tests are disabled
2019-06-26 14:40:52 -07:00
Yin Guanhao 6316aa1d0b Update tokio-udp to use std-future (#1199) 2019-06-26 14:41:36 -04:00
Bhargav 0784dc2767 tokio: add read_exact method (#1202) 2019-06-26 11:36:09 -07:00
Denis dd126c2333 Implement TryFrom to transform various I/O primitives into their mio counterparts (#1158)
* `TryFrom<TcpListener> for mio::net::TcpListener`
* `TryFrom<TcpStream> for mio::net::TcpStream`
* `TryFrom<UdpSocket> for mio::net::UdpSocket`
* `TryFrom<UnixListener> for mio_uds::UnixListener`
* `TryFrom<UnixStream> for mio_uds::UnixStream`
2019-06-26 08:51:38 -07:00
Lucio Franco 3cc33dca7c sync: Fix lock test to actually test the inner lock value (#1197)
* sync: Fix lock test to actually test the returned value

* Update lock test to use task.poll
2019-06-26 11:32:41 -04:00
Carl Lerche dc5fa80a09 macros: re-export main macro from tokio (#1198)
Includes minor fixes and a very basic example.

Fixes #1183
2019-06-25 20:14:21 -07:00
Zahari Dichev 455782b964 trace: Allow setting event parents explicitly (#1109)
## Motivation 

As mentioned in tokio-rs/tracing#1100  it makes sense to be able to set
the parents of events explicitly.

## Solution 

For that to happen the Parent type is extracted from span.rs and a
`parent` field is added to Event. Additionally the appropriate macros
arms are added with corresponding tests as described in
tokio-rs/tracing#1100

Closes tokio-rs/tracing#1100

Signed-off-by: Zahari Dichev <[email protected]>
2019-06-25 15:12:52 -07:00
Lucio Franco 29e417c257 tokio: Add io copy, read, and write (#1187) 2019-06-25 16:51:49 -04:00
Ivan Petkov 9df1140340 signal: factor out event delivery into its own module to share between Unix and Windows (#1174)
Today the Unix and Windows implementations have similar yet differing
implementations of hooking into OS events and propagating them to any
listening futures. Rather than re-implement the same behavior two
different ways, we should factor out any commonality into a shared
module and keep the Unix/Windows modules focused solely on OS
integrations.

Reusing the same implementation across OS versions also allows for more
consistent behavior between platforms, which also makes squashing bugs
much easier.

This change introduces the `registry` module which handles creating and
initializing a global map of signals/events and their registered
listeners. Each OS specific module is expected to implement the OS hooks
which delegate to invoking the registry module's methods for
distributing the event notifications.

# Use registry module for Windows implementation

Note this still uses the same architecture as previously: a driver task
is spawned by the first registered event, and that task is responsible
for delivering any events to registered futures. (If that first event
loop goes away, all events will deadlock). A solution to this issue will
be explored at a later time.
2019-06-25 13:07:59 -07:00
Lucio Franco e2b4bdb647 sync: Add LockFuture for Lock (#1184) 2019-06-25 10:42:35 -07:00
Ivan Petkov c6defbce4b process: Move files to their own directory 2019-06-24 17:31:47 -07:00
Ivan Petkov b7846a4e2f process: Remove unneeded files 2019-06-24 17:31:00 -07:00
Ivan Petkov cb8607a816 process: Update to 2018 edition 2019-06-24 17:29:33 -07:00
Ivan Petkov 27c15471c1 process: Run cargo fmt 2019-06-24 17:29:33 -07:00
Ivan Petkov 0ab25878bd process: Update README 2019-06-24 17:29:32 -07:00
Eliza Weisman 448302c3d4 trace: Improve documentation (#1148) 2019-06-24 19:22:05 -05:00
Ivan Petkov 934a1467d4 process: Update CHANGELOG 2019-06-24 17:12:17 -07:00
Ivan Petkov 4d639e246b process: Update Cargo.toml 2019-06-24 17:10:58 -07:00
Ivan Petkov ff5381de8d process: Update license files 2019-06-24 17:10:58 -07:00
Ivan Petkov 061452dc01 process: Delete flaky and (now) unused test 2019-06-24 17:10:58 -07:00
Ivan Petkov a6b2682309 process: Bump to 0.2.4 2019-06-24 16:57:20 -07:00
Ivan Petkov cf84a59e5a process: Don't kill child on drop if already successfully killed 2019-06-24 16:57:20 -07:00
Ivan Petkov e90e33d5df process: Add unit tests for dropping killing dropped children 2019-06-24 16:57:20 -07:00
Ivan Petkov fa5da27d98 process: Utilize a global orphan process queue to avoid leaks 2019-06-24 16:57:20 -07:00
Ivan Petkov ecaa069f0f process: Implement a queue for repeatedly attempting to reap orphaned processes 2019-06-24 16:57:20 -07:00
Ivan Petkov fc15d7d4a4 process: Only pull in mio dependency on unix platforms 2019-06-24 16:57:20 -07:00
Ivan Petkov a70a3b599a process: ci: move cargo tool installation to after_success 2019-06-24 16:57:20 -07:00
Ivan Petkov 26faefcc34 process: ci: enable clippy checks as part of the build 2019-06-24 16:57:19 -07:00
Ivan Petkov f16725ea9f process: Fix clippy warnings 2019-06-24 16:57:19 -07:00
Ivan Petkov caf43221b5 process: ci: fix cargo binary caching 2019-06-24 16:57:19 -07:00
Ivan Petkov 93680357dd process: Fix drop_kills test when running on macOS with a single thread 2019-06-24 16:57:19 -07:00
Ivan Petkov 784d21ae31 process: Try pinning mio to 0.1.16 2019-06-24 16:57:19 -07:00
Ivan Petkov 0938ccfefd process: ci: cache cargo tarpaulin build 2019-06-24 16:57:19 -07:00
Ivan Petkov 6fa2fdab44 process: Ensure all tests are run with an explicit timeout 2019-06-24 16:57:19 -07:00
Ivan Petkov d0d13d0bd0 process: Change codecov comment behavior to default 2019-06-24 16:57:19 -07:00
Ivan Petkov 8a1777b800 process: Rename EventedReaper to Reaper 2019-06-24 16:57:18 -07:00
Ivan Petkov 42d0f53ddb process: Optimize out the "reaped" flag 2019-06-24 16:57:18 -07:00
Ivan Petkov db0c4147c8 process: Refactor Unix process handling 2019-06-24 16:57:18 -07:00
Ivan Petkov 10fd2afd18 process: Simplify child IO registration 2019-06-24 16:57:18 -07:00
Ivan Petkov 83a55601ef process: Move src/unix.rs to src/unix/mod.rs 2019-06-24 16:57:18 -07:00
Ivan Petkov b37120f61c process: Update line-by-line doc example to be more flexible 2019-06-24 16:57:18 -07:00
Ivan Petkov 91dbf24cf4 process: Update min supported rust version as per the Tokio project policy 2019-06-24 16:57:18 -07:00
Ivan Petkov c78fd6d6c5 process: Update Travis link from .org to .com 2019-06-24 16:57:18 -07:00
Ivan Petkov 025474dfbb process: ci: Install cargo-tarpaulin *after* initial tests 2019-06-24 16:57:18 -07:00
Ivan Petkov e7dfcf90fe process: ci: Enable code coverage tracking via codecov.io 2019-06-24 16:57:17 -07:00
Ivan Petkov ecdfe4c474 process: ci: collect code coverage info via cargo-tarpaulin 2019-06-24 16:57:17 -07:00
Ivan Petkov 37b4efb9e2 process: Bump version to 0.2.3 2019-06-24 16:57:17 -07:00
Ivan Petkov c94f607f1b process: Fix some test case deprecation warnings 2019-06-24 16:57:17 -07:00
Ivan Petkov 76438c9e70 process: Implement AsRawHandle for ChildStd{in, out, err} for parity 2019-06-24 16:57:17 -07:00
Yuya Nishihara e0e9594f71 process: Implement AsRawFd for ChildStd* structs 2019-06-24 16:57:17 -07:00
Yuya Nishihara 3b43262a10 process: Implement AsRawFd for inner Fd<T> wrappers and use it instead of self.0 2019-06-24 16:57:17 -07:00
Ivan Petkov 5f18bf669f process: Bump minimum supported rustc version to 1.26 2019-06-24 16:57:17 -07:00
Ivan Petkov 1581c8b475 process: Bump minimum required version of tokio-signal to 0.2.5 2019-06-24 16:57:16 -07:00
Ivan Petkov d3b2efc815 process: Add regression test for signal starvation 2019-06-24 16:56:53 -07:00
Ivan Petkov f7c4e3cd84 process: Bump min supported rustc version to 1.25 2019-06-24 16:56:53 -07:00
Ivan Petkov 329ad3324c process: Bump to 0.2.2 2019-06-24 16:56:53 -07:00
Ivan Petkov 2b6695d25a process: Update CHANGELOG 2019-06-24 16:56:53 -07:00
Ivan Petkov 9290602815 process: Unix: preregister for signal notifications before polling child 2019-06-24 16:56:53 -07:00
Ivan Petkov 827e77e71e process: Bump to 0.2.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 7b3e4b98ac process: Update Child::forget example to use the tokio runtime 2019-06-24 16:56:52 -07:00
Ivan Petkov 5e9d60e834 process: Add a CHANGELOG 2019-06-24 16:56:52 -07:00
Ivan Petkov 8270965459 process: Remove dependency on tokio-core 2019-06-24 16:56:52 -07:00
Ivan Petkov e6b044a820 process: Bump tokio-signal version to 0.2 2019-06-24 16:56:52 -07:00
Ivan Petkov de9b401457 process: Mark status_async2/StatusAsync2 as deprecated 2019-06-24 16:56:52 -07:00
Ivan Petkov ad5179b2d5 process: Remove all items deprecated in 0.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 0aceba21bd process: Bump to 0.1.6 2019-06-24 16:56:52 -07:00
Ivan Petkov 09e21eceea process: Unix: mark child as reaped on kill 2019-06-24 16:56:52 -07:00
Arvid E. Picciani bdc87856f2 process: fix zombification on Drop on unix 2019-06-24 16:56:51 -07:00
Ivan Petkov 7987b64445 process: Clarify that Child::forget docs that it can leak OS resources 2019-06-24 16:56:51 -07:00
Alex Crichton 32c928b607 process: Bump to 0.1.5 2019-06-24 16:56:51 -07:00
Alex Crichton 82aeae147d process: Update dev-dependencies 2019-06-24 16:56:51 -07:00
Alex Crichton f48944c1fb process: Update winapi to 0.3 2019-06-24 16:56:51 -07:00
Ivan Petkov f0680617ee process: Fix project name typo in README 2019-06-24 16:56:51 -07:00
Alex Crichton dbc185cd3a process: Tweak travis config 2019-06-24 16:56:51 -07:00
Alex Crichton c205e2c358 process: Fix copy/paste 2019-06-24 16:56:51 -07:00
Alex Crichton acec6356ee process: Clarify wording of license information in README. 2019-06-24 16:56:51 -07:00
Alex Crichton c11eec3908 process: Bump to 0.1.4 2019-06-24 16:56:50 -07:00
Alex Crichton 69295fac1e process: Add an Errors section to status_async2 2019-06-24 16:56:50 -07:00
Ivan Petkov b9c6eb309c process: Add status_async2 as a closer analog to spawn_async 2019-06-24 16:56:50 -07:00
Ivan Petkov 56d3914675 process: Bugfix: ensure status_async closes child's stdio handles after spawning 2019-06-24 16:56:50 -07:00
Ivan Petkov 34e71fa71a process: Add must_use annotations to all futures 2019-06-24 16:56:50 -07:00
Ivan Petkov 914b803429 process: Add Debug impls for nondeprecated structs 2019-06-24 16:56:50 -07:00
Alex Crichton 4d11784b01 process: Tweak docs and macro imports 2019-06-24 16:56:50 -07:00
Michael Pankov 50cabae181 process: Add an example with reading input line-by-line 2019-06-24 16:56:50 -07:00
Alex Crichton c101e9e11d process: Use appveyor to download rustup 2019-06-24 16:56:49 -07:00
Alex Crichton 5c5f793ef0 process: Bump to 0.1.3 2019-06-24 16:56:49 -07:00
Alex Crichton 1384b31d60 process: Update to tokio-io, mio, and tokio-core changes 2019-06-24 16:56:49 -07:00
Alex Crichton 521dc94021 process: Bump to 0.1.2 2019-06-24 16:56:49 -07:00
Alex Crichton ed23a06fb1 process: Update doc urls and metadata 2019-06-24 16:56:49 -07:00
Alex Crichton 1aee22505a process: Remove caveat about tokio-signal 2019-06-24 16:56:49 -07:00
Alex Crichton 01b5bf6761 process: Use join3 instead of two joins 2019-06-24 16:56:49 -07:00
Alex Crichton 6638cbc80e process: Update README 2019-06-24 16:56:49 -07:00
Alex Crichton 22bc5e2738 process: Bump back to 0.1.1 2019-06-24 16:56:49 -07:00
Alex Crichton a0c162c0ff process: Hide compat from docs 2019-06-24 16:56:48 -07:00
Alex Crichton ca51ae9651 process: Add back in 0.1.0 compatibility layer 2019-06-24 16:56:48 -07:00
Alex Crichton f3f99b723f process: Bump to 0.2.0 2019-06-24 16:56:48 -07:00
Alex Crichton f20e7a4d2b process: Bump minimum version of tokio-core 2019-06-24 16:56:48 -07:00
Alex Crichton 4a92c4d4b6 process: Tweak drop_kills test 2019-06-24 16:56:48 -07:00
Alex Crichton 9680ecc109 process: Share init in tests 2019-06-24 16:56:48 -07:00
Alex Crichton 124391e42b process: Add a simple wait_with_output test 2019-06-24 16:56:48 -07:00
Alex Crichton 6150be189f process: Rewrite the crate with an extension trait 2019-06-24 16:56:48 -07:00
Ivan Petkov ca9586a089 process: Add documentation to public declarations 2019-06-24 16:56:47 -07:00
Ivan Petkov 89b9792931 process: Update README with crates.io info 2019-06-24 16:56:47 -07:00
Alex Crichton a0cc60153a process: Fix nightly tests 2019-06-24 16:56:47 -07:00
Alex Crichton 7f3f868b66 process: Add Windows support for stdio streams 2019-06-24 16:56:47 -07:00
Andreas Rottmann 97ebb2275c process: [WIP] Actually be non-blocking 2019-06-24 16:56:47 -07:00
Andreas Rottmann 849a5ad0b2 process: Add support for stdio streams 2019-06-24 16:56:47 -07:00
Alex Crichton b16a8613b1 process: Test on stable 2019-06-24 16:56:47 -07:00
Alex Crichton 5664660156 process: Fix tests on nightly 2019-06-24 16:56:47 -07:00
Alex Crichton 4416ea07d8 process: Update travis token 2019-06-24 16:56:47 -07:00
Alex Crichton 56222c588b process: pass --target on appveyor 2019-06-24 16:56:46 -07:00
Alex Crichton 72179d49c5 process: Update to crates.io versions of deps 2019-06-24 16:56:46 -07:00
Alex Crichton 073a1a251a process: Track tokio-core master 2019-06-24 16:56:46 -07:00
Alex Crichton 31c81faf96 process: Add appveyor to readme 2019-06-24 16:56:46 -07:00
Alex Crichton 5e68b0d51d process: Don't build on stable, start w/ beta for now 2019-06-24 16:56:46 -07:00
Alex Crichton 4bd07ac6aa process: Add metadata info 2019-06-24 16:56:46 -07:00
Alex Crichton f4f7bb232e process: Fix a test on Windows 2019-06-24 16:56:46 -07:00
Alex Crichton 413e1b78a7 process: Fix a segfault on windows 2019-06-24 16:56:46 -07:00
Alex Crichton 649fa13a15 process: Remove unused imports 2019-06-24 16:56:45 -07:00
Alex Crichton eef655f3b1 process: Add a Windows implementation 2019-06-24 16:56:45 -07:00
Alex Crichton 97508096fa process: Initial commit 2019-06-24 16:56:41 -07:00
Carl Lerche 06c473e628 Update Tokio to use std::future. (#1120)
A first pass at updating Tokio to use `std::future`.

Implementations of `Future` from the futures crate are updated to implement
`Future` from std. Implementations of `Stream` are moved to a feature flag.

This commits disables a number of crates that have not yet been updated.
2019-06-24 12:34:30 -07:00
James Gilles aa99950b9c trace: Switch benchmarks to criterion (#1163)
Extracted from #1152

This makes it possible to run benchmarks on stable + gives more statistical reliability.
2019-06-24 12:05:45 -07:00
Takanori Ishibashi aac6998c22 chore: fix url in docs (#1173) 2019-06-24 07:33:46 -04:00
Matt Bilker df2c3cd475 trace: fix debug and debug_span macro regression from #1103 (#1170)
PR #1103 accidentally changed the log level for the debug and
debug_span macros to use the INFO level instead of the DEBUG
level. This PR corrects this regression back to the intended
behavior.
2019-06-22 16:31:25 -07:00
James Gilles 36ed35c52c trace: add program-wide default dispatcher (#1152)
## Motivation

I was just trying to use tokio-trace for a greenfield project, but I was frustrated to discover that I couldn't really use it easily.

I was using the [`runtime`](https://docs.rs/runtime/0.3.0-alpha.4/runtime/) crate, which transparently spawns a thread pool executor for futures. In that thread pool, there's no way to set a tokio-trace subscriber for the duration of each thread, since you don't control the thread initialization. You *might* be able to wrap every future you spawn with a subscriber call, but that's a lot of work.

I was also confused because the documentation said that setting a subscriber in the main thread would use that subscriber for the rest of the program. That isn't the case, though -- the subscriber will be used only on the main thread, and not on worker threads, etc.

## Solution

I added a function `set_global_default`, which works similarly to the `log` crate:

```rust
tokio_trace::subscriber::set_global_default(FooSubscriber::new());
```

The global subscriber (actually a global `Dispatch`) is a `static mut` protected by an atomic; implementation is copied from the `log` crate. It is used as a fallback if a thread has no `Dispatch` currently set. This is extremely simple to use, and doesn't break any existing functionality.

Performance-wise, thread-local `Dispatch` lookup goes from ~4.5ns to ~5ns, according to the benchmarks. So, barely any runtime overhead. (Presumably there's a little compile-time overhead but idk how to measure that.) Since the atomic guard is only ever written once, it will be shared among a CPU's cores and read very cheaply.

I added some docs to partially address #1151. I also switched the tokio-trace benchmarks to criterion because the nightly benchmarks weren't compiling (missing `dyn` flags?)
2019-06-21 16:49:53 -07:00
Eliza Weisman 5925ca7720 trace: fix level_span macros not propagating parents (#1167)
Currently, when the `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!`, and `error_span!` macros are invoked with an explicit
parent, a name, and zero or more fields (no target), the macros don't
pass along the explicitly provided parent when expanding to the `span!`
macro. This is likely due to an oversight on my part.

This branch fixes these macros by adding the parent into the `span!`
macro expansion. I've also added a test to catch regressions

Shoutout to @jonhoo for catching this one!

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-21 11:17:06 -07:00
Max Bruckner 2ac132fb46 runtime: better error message in block_on_all on panics (#1166) 2019-06-21 11:10:58 -04:00
Hung-I Wang f9a0cb8792 timer: Implement Default for DelayQueue (#1118) 2019-06-21 10:42:52 -04:00
Igor Gnatenko 9fa6092e5a chore: Update parking_lot to 0.8 (#1078) 2019-06-21 10:42:09 -04:00
Eliza Weisman d4adeeef2f trace: Remove the AsId trait (#1145)
While we're making breaking changes to `tokio-trace`, it would be good
to get rid of the `AsId` trait. The goal of span functions that are
generic over `Span`/`Id` can be achieved without the unnecessary
complexity of defining a new trait. This would also make the API added
to `tokio_trace_core::Event` in #1109 more consistent with the
`tokio-trace::Span` API.

This branch removes `AsId` from `tokio-trace` and replaces its uses with
`impl Into<Option<Id>>` and `impl Into<Option<&'a Id>>`. While `AsRef`
might be more semantically correct for the borrowed-`Id` conversion, its
signature doesn't permit conversion into an `Option`. Implementations of
`Into<Option<Id>>` and `Into<Option<&'a Id>>` have been added for 
`tokio_trace::Span`.

This is _technically_ a breaking API change, as it changes function
signatures. However, the existing macro syntax still works as-is, and
the tests which pass `&Id`, `&Span`, and `&Option<Id>` to the span
macros all still compile after this change.

Closes #1143

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-13 12:53:08 -07:00
Steven Fackler 4f6395b31c Make threadpool::Runtime methods take &self (#1140)
The runtime is inherently multi-threaded, so it's going to have to deal
with synchronization when submitting new tasks anyway. This allows a
runtime to be shared by multiple threads more easily when e.g. building
a blocking facade over a tokio-based API.
2019-06-10 12:54:27 -07:00
yanjhk 5c0b56278b Use ThreadPool's impl of spawn (#1139) 2019-06-10 11:23:12 -07:00
Eliza Weisman 41ca9a43de trace: Add shorthand syntax for local fields (#1103)
## Motivation

A common pattern in `tokio-trace` is to use the value of a local
variable as a field on a span or event. Currently, this requires code
like:
```rust
info!(foo = foo);
```
which is not particularly ergonomic given how commonly this occurs.
Struct initializers support a shorthand syntax for fields where the name
of the field is the same as a local variable, and `tokio-trace` should
as well.

## Solution

This branch adds support for syntax like
```rust
let foo = ...;
info!(foo);
```
and 
```rust
let foo = Foo {
    bar: ...,
    ...
};
info!(foo.bar)
```
to the `tokio-trace` span and event macros. This syntax also works with
the `Debug` and `Display` field shorthand.

The span macros previously used a field name with no value to indicate 
an uninitialized field. A new issue, #1138, has been opened for finding a
replacement syntax for uninitialized fields. Until then, the `tokio-trace` 
macros will no longer provide a way to create fields without values, 
although the `-core` API will continue to support this.

Closes #1062 

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-09 13:16:35 -07:00
Carl Lerche 8d0f102de8 Merge branch 'v0.1.x' into merge-0.1 2019-06-05 12:28:39 -07:00
Kevin Leimkuhler 5dcb379f6d Bump tokio-sync to 0.1.6 (#1123) 2019-06-05 12:19:06 -07:00
Kevin Leimkuhler 970f75f830 sync: Add Sync impl for Lock (#1117) 2019-06-04 17:04:35 -07:00
Kevin Leimkuhler 619efed28b sync: Add Sync impl for Lock (#1116)
Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-06-03 11:12:28 -07:00
Carl Lerche 18ed0be851 executor: remove unnecessary APIs from Enter. (#1115) 2019-05-31 11:11:10 -07:00
Carl Lerche 01052f930a Bump tokio version to v0.1.21. (#1113) 2019-05-30 14:39:30 -07:00
Lucio Franco 940f2c3431 Update tokio-trace-core to 0.2 (#1111)
Also includes 1b498e8aa2
2019-05-30 11:33:55 -07:00
Eliza Weisman 84d5a7f5a0 trace: Change Span::enter to return a guard, add Span::in_scope (#1076)
## Motivation

Currently, the primary way to use a span is to use `.enter` and pass a
closure to be executed under the span. While that is convenient in many
settings, it also comes with two decently inconvenient drawbacks:

 - It breaks control flow statements like `return`, `?`, `break`, and
   `continue`
 - It require re-indenting a potentially large chunk of code if you wish
   it to appear under a span

## Solution

This branch changes the `Span::enter` function to return a scope guard 
that exits the span when dropped, as in:
```rust
let guard = span.enter();

// code here is within the span

drop(guard);

// code here is no longer within the span
```
The method previously called `enter`, which takes a closure and 
executes it in the span's context, is now called `Span::in_scope`, and
was reimplemented on top of the new `enter` method. 

This is a breaking change to `tokio-trace` that will be part of the
upcoming 0.2 release.

Closes #1075 

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-24 15:24:13 -07:00
Carl Lerche 1b498e8aa2 Fix TCP poll_hup test (#1106)
This updates tests to track a fix applied in Mio. Previously, Mio
incorrectly fired HUP events. This was due to Mio mapping `RDHUP` to
HUP. The test is updated to correctly generate a HUP event.

Additionally, HUP events will be removed from all platforms except for
Linux. This is caused by the inability to reliably map kqueue events to
the epoll HUP behavior.
2019-05-24 14:08:07 -07:00
Eliza Weisman b2c53987d9 trace: Add shorthand for field::display and field::debug (#1088)
## Motivation

In `tokio-trace`, field values may be recorded as either a subset of
Rust primitive types or as `fmt::Display` and `fmt::Debug`
implementations. Currently, `tokio-trace` provides the `field::display`
and `field::debug` functions which wrap a type with a type that
implements `Value` using the wrapped type's `fmt::Display` or
`fmt::Debug` implementation. However, importing and using these
functions adds unnecessary boilerplate. 

In #1081, @jonhoo suggested adding shorthand syntax to the macros,
similar to that used by the `slog` crate, as a solution for the
wordiness of the current API.

## Solution

This branch adds `?` and `%` sigils to field values in the span and
event macros, which expand to the `field::debug` and `field::display`
wrappers, respectively. The shorthand sigils may be used in any position
where the macros take a field value.

For example:
```rust
trace_span!("foo", my_field = ?something, ...); // shorthand for `debug`
info!(foo = %value, bar = false, ...) // shorthand for `display`
```

Adding this shorthand required a fairly large change to how field
key-value pairs are handled by the macros --- since `%foo` and `%foo`
are not valid Rust expressions, we can no longer match repeated 
`$ident = $expr` patterns, and must now match field lists as repeated
token trees. The inner helper macros for constructing `FieldSet`s and
`ValueSet`s have to parse the token trees recursively. This added a
decent chunk of complexity, but fortunately we have a large number of
compile tests for the macros and I'm quite confident that all existing
invocations will still work.

Closes #1081

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-21 10:31:48 -07:00
Carl Lerche 38092010c4 Merge branch 'v0.1.x' 2019-05-14 11:50:44 -07:00
Carl Lerche 475dabe96d Release tokio v0.1.20, tokio-timer v0.2.21, and remove async-await-preview feature. (#1089)
The `async-await-preview` feature is removed as 0.1 will no longer track
Rust nightly.

This also bumps:
- tokio-timer (0.2.11).
2019-05-14 11:21:24 -07:00
Carl Lerche cb4aea394e Update Tokio to Rust 2018 (#1082) 2019-05-14 10:27:36 -07:00
Jeehoon Kang 79d8820050 Fix link in tokio-futures/README.md (#1085)
`tokio-futures/README.md`'s link to the examples was wrong.
2019-05-10 10:19:32 -07:00
Carl Lerche 951f2fd910 test: re-export macro dependencies (#1077)
Callers may not always have `futures` available at the root of the
crate. Re-exporting dependencies makes them available to the macro at a
deterministic location.
2019-05-03 20:43:40 -07:00
Carl Lerche 4ef736b9d5 async-await: add current_thread::Runtime::block_on_async (#1072)
This function is used by the Tokio macros introduced by #1058  but was
omitted from the PR.
2019-04-30 19:55:22 -07:00
Steven Fackler 219f24cbf1 timer: Replace Handle::deadline with Handle::timeout (#1074)
Deadline was deprecated a while ago and replaced with Timeout, but the
methods on Handle got missed.

Fixes #1071
2019-04-30 10:29:54 -07:00
Carl Lerche ea282efb2e ci: fix isRelease condition (#1066) 2019-04-29 10:37:00 -07:00
Michal 'vorner' Vaner 042224d33c signal: Smaller dependency (#1069)
The signal-hook library got split into lower-level and higher-level
parts. The tokio-signal uses only API from the lower-level one, so it
can depend on it directly.

The only effect of this change is smaller amount of compiled (and
unused) code during compilation. There's no change in the code actually
used.
2019-04-28 19:12:40 -07:00
Ian Hamlin 927eb80ad4 Fix an error in the mit-url in the README.md (#1068) 2019-04-27 12:56:40 -07:00
Carl Lerche 6a8934e897 Fix threadpool dependency (#1061) 2019-04-25 22:23:24 -04:00
Carl Lerche 0e400af78c Async/await polish (#1058)
A general refresh of Tokio's experimental async / await support.
2019-04-25 22:22:32 -04:00
Carl Lerche df702130d6 tcp: fix some tests that spuriously fail (#1060)
This does not remove all cases of using a fixed port in doc tests, but
removing some should reduce the likelihood of spurious failures.
2019-04-25 12:01:39 -07:00
Carl Lerche 949adbb887 chore: remember to remove path deps on release (#1057) 2019-04-24 10:42:39 -07:00
Ryan Dahl b2b796a228 rt: forward panic_handler to tokio::runtime::Builder (#1055) 2019-04-24 10:41:42 -07:00
Igor Gnatenko abb014efc2 tokio: Bump min version of tokio-sync (#1054)
It is needed for lock functionality which tokio now uses.
2019-04-24 08:24:59 -07:00
Lucio Franco e5cf0cc717 Introduce tokio-test crate (#1030) 2019-04-23 20:17:57 -07:00
Carl Lerche 62f34e15ce Bump tokio to 0.1.19. (#1053)
This also bumps:

- tokio-async-await (0.1.7)
- tokio-buf (0.1.1)
- tokio-sync (0.1.5)
- tokio-threadpool (0.1.14)
2019-04-22 15:12:25 -07:00
Eliza Weisman 3ebca76a9a trace: prepare tokio-trace for release (#1051) 2019-04-22 14:15:07 -07:00
Ryan Dahl fea1f780bc threadpool: add panic_handler (#1052) 2019-04-21 16:26:09 -07:00
Eliza Weisman 712ca84033 trace-core: prepare for 0.2 release (#1047) 2019-04-21 10:20:28 -07:00
Jon Gjengset cf06621998 tokio-sync: Add async mutual exclusion primitive (#964)
This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.

This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
2019-04-18 13:16:26 -04:00
Lucio Franco 7e51ab05e9 buf: Add IntoStream (#1048)
* buf: Add IntoStream

* Add debug implementation for IntoStream

* Add get_ref, get_mut and into_inner
2019-04-18 11:42:34 -04:00
Eliza Weisman 4bfa4ffcdf trace-core: Dispatchers unset themselves (#1033)
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans. 

Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.

Before:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          51 ns/iter (+/- 12)
test span_repeatedly         ... bench:       5,073 ns/iter (+/- 1,528)
test span_with_fields        ... bench:          56 ns/iter (+/- 49)
test span_with_fields_record ... bench:         363 ns/iter (+/- 61)
```

After:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          35 ns/iter (+/- 12)
test span_repeatedly         ... bench:       4,165 ns/iter (+/- 298)
test span_with_fields        ... bench:          48 ns/iter (+/- 12)
test span_with_fields_record ... bench:         363 ns/iter (+/- 91)
```

Closes #1032 

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-16 15:51:45 -07:00
Taiki Endo 88b942652c async-await: fix examples (#1050)
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
  on Rust 2018
* Fix an example in the documentation
2019-04-16 14:19:38 -04:00
Taiki Endo 5029e80a89 async-await: update to new futures_api (#1049) 2019-04-16 14:07:03 -04:00
Eliza Weismanandcsmoe 847fb59b17 trace-core: Introduce callsite classification in metadata (#1046)
## Motivation

To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.

## Solution

A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.

`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.

New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.

Fixes: #986
Closes: #1008

Co-Authored-By: csmoe <[email protected]>
2019-04-11 11:56:42 -07:00
Jane Lusby b4fe517a16 trace-core: add a function to rebuild cached interest (#1039)
## Motivation

Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.

However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`.  Even when filters change very
infrequently, they must still always be re-evaluated every time.

In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.

## Solution

This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.

## Breaking Change

In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.

Closes #1038

Co-Authored-By: yaahallo <[email protected]>
2019-04-10 13:51:05 -07:00
Simon Wollwage 7ae010f0f3 async-await: Use Context instead of Waker in poll (#1041)
Rust nightly std::future::Future recently changed Waker
to Context.

Change to use Context

Co-Authored-By: Kintaro <[email protected]>
2019-04-10 09:14:31 -07:00
Carl Lerche 9144b2ff53 sync: remove unnecessary imports (#1043) 2019-04-09 12:26:11 -07:00
Lev Eniseev 2c4549a18a Add example of blocking environment (#1036) 2019-04-09 12:10:15 -07:00
João Oliveira 4f819b7cd1 trace: fix counters example, Span IDs must be greater than zero (#1037)
## Motivation
tokio-trace counter example was panicking due to returning 0 as the first Span Id

## Solution
start ID's with 1 instead
2019-04-05 13:51:20 -07:00
Eliza Weisman 197f88f3bc trace: Change Span::enter and record to take &self (#1029)
## Motivation

The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.

Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
    span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
    span.record("bar" &false);
});
```

Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.

The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.

## Solution

This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows. 

In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`. 

## Notes

In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.

Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.

Before:
```
test enter_span              ... bench:          13 ns/iter (+/- 1)
```

...and after:
```
test enter_span              ... bench:           3 ns/iter (+/- 1)
```

Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.

I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 15:06:47 -07:00
Eliza Weisman 44f65afcc6 trace: Allow field names to be separated by .s (#1027)
## Motivation

In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.

## Solution

This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.

## Notes

The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.

Closes #1018.
Closes #1022.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 14:44:11 -07:00
Eliza Weisman 4271a9cd8d trace: Fix subscriber benchmarks panicking (#1028)
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-02 13:42:31 -07:00
Matthias Prechtl 9d8096b911 Improve documentation of Subscriber::record and Subscriber::event (#1026) 2019-04-02 12:44:01 -07:00
João Oliveira 597f271c08 trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation 

Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952). 

Closes #1013

## Solution 

Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels

## Notes 

I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
2019-04-02 11:29:23 -07:00
Taiki Endo 599955f716 Replace try! macro with ? operator (#1024) 2019-04-01 13:45:59 -07:00
Ivan Petkov 91bb0f73f5 signal: refactor Windows registrations to be lazy (#1001)
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
  so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
  when calling `global_init`

Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS

Fixes #999
2019-04-01 12:46:22 -07:00
Eliza Weisman 6c9d8abba9 trace: Make Span API functions taking IDs a little more flexible (#1021)
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.

Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-01 11:42:29 -07:00
Carl Lerche 824b7b6759 buf: stream and iter helpers (#1011) 2019-03-29 12:26:13 -07:00
Carl Lerche cb91dd274a buf: impl Error for CollectVecError (#1010) 2019-03-29 08:49:08 -07:00
Eliza Weisman ea7178b8c6 trace-core: Add slightly more useful debug impls (#1014)
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.

When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.

The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-28 14:47:06 -07:00
Carl Lerche a99b8e2e0b buf: impl size_hint for str types + reorg tests (#1012) 2019-03-28 14:02:45 -07:00
Carl Lerche 03859a7dcd buf: implement FromBufStream for Bytes (#1009) 2019-03-27 19:22:06 -07:00
Eliza Weisman d8177f81ac trace: Allow trace instrumentation to emit log records (#992)
## Motivation

`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.

Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.

## Solution

This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation. 

## Notes

In order to allow spans to log their names when they are entered and 
exited even when the span is disabled, this branch adds an 
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.

Fixes: #949

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-26 16:43:05 -07:00
Son ceca2a3cd6 chore: add license to tokio (#1006) 2019-03-26 08:41:41 -07:00
Son 1524ee4b60 trace: Add static level filtering (#987)
## Motivation

`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.

## Solution

We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.

Closes #959
2019-03-25 15:19:20 -07:00
MOZGIII 7793d63739 Corrected doc for tokio_buf SizeHint (#1003) 2019-03-25 16:54:49 -04:00
Red Hara e0e26bc223 Fix typo in README.md in examples (#1002) 2019-03-24 13:02:11 -04:00
Eliza Weisman 9c5cad037f trace-core: Add overrideable downcasting to Subscribers (#974)
## Motivation

In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to 
downcast the current `Dispatch` to a concrete subscriber type.

For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust

pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
    tokio_trace::dispatcher::with(|c| {
        if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
            s.do_elizatracing_correlation_magic(tt, et);
        }
    }); 
}
```

This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).

## Solution

This branch makes the following changes:
 * Add an object-safe `downcast_raw` method to the `Subscriber` trait,
   taking a `TypeId` and returning an `*const ()` if the type ID 
   matches the subscriber's type ID, or `None` if it does not, and
 * Add `is<T>` and `downcast_ref<T>` functions to `Subscriber` 
   and `Dispatch`, using `downcast_raw`.

Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_ 
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour 
somehow, can downcast to the inner type if they choose to.

[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204

Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 16:21:46 -07:00
Eliza Weisman 30330da11a chore: Fix examples not working with cargo run (#998)
* chore: Fix examples not working with `cargo run`

## Motivation

PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:

```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
error: no example target named `chat`

Did you mean `echo`?
```

## Solution

This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:

```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
   Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
   Compiling tokio-reactor v0.1.9
   Compiling tokio-threadpool v0.1.13
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.10
   Compiling tokio-uds v0.2.5
   Compiling tokio-udp v0.1.3
   Compiling tokio-tcp v0.1.3
   Compiling tokio-fs v0.1.6
   Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
    Finished dev [unoptimized + debuginfo] target(s) in 7.04s
     Running `target/debug/examples/chat`
server running on localhost:6142
```

Signed-off-by: Eliza Weisman <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 15:25:42 -07:00
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
719 changed files with 43300 additions and 43674 deletions
-22
View File
@@ -1,22 +0,0 @@
image: Visual Studio 2017
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- set RUST_BACKTRACE=1
- set LOOM_MAX_DURATION=10
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --no-fail-fast --target %TARGET%
+42
View File
@@ -0,0 +1,42 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_PREEMPTIONS: 2
RUSTFLAGS: -Dwarnings
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y --profile minimal --default-toolchain stable
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
test_script:
- . $HOME/.cargo/env
- cargo test --all
- cargo doc --all --no-deps
# TODO: Re-enable
# i686_test_script:
# - . $HOME/.cargo/env
# - |
# cargo test --all --exclude tokio-tls --exclude tokio-macros --target i686-unknown-freebsd
-131
View File
@@ -1,131 +0,0 @@
---
language: rust
sudo: false
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
- rust: stable
- rust: beta
- rust: nightly
env: ALLOW_FAILURES=true
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
#
# Tests are not run as tests may require newer versions of rust.
- rust: 1.26.0
script: |
cargo check --all
# Test combinations of enabled features.
- rust: stable
script: |
shopt -s expand_aliases
alias check="cargo check --no-default-features"
check
check --features codec
check --features fs
check --features io
check --features reactor
check --features rt-full
check --features tcp
check --features timer
check --features udp
check --features uds
# Test the async / await preview. We don't want to block PRs on this failing
# though.
- rust: nightly
env: ALLOW_FAILURES=true
script: |
cd tokio-async-await
cargo check --all
# This runs TSAN against nightly and allows failures to propagate up.
- rust: nightly-2018-11-18
env: TSAN=yes
script: |
set -e
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# This runs cargo +nightly doc
- name: nightly_docs
rust: nightly
env: ALLOW_FAILURES=true
script: cargo doc
allow_failures:
- rust: nightly
env: ALLOW_FAILURES=true
script: |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --exclude tokio-tls --target $TARGET
cargo check --tests --all --exclude tokio-tls --target $TARGET
else
# Limit the execution time of loom tests.
export LOOM_MAX_DURATION=10
cargo test --all --no-fail-fast
cargo doc --all
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = "linux" && $TARGET = ""
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+61 -5
View File
@@ -12,7 +12,7 @@ use your help.
This guide will help you get started. **Do not let this guide intimidate you**.
It should be considered a map to help you navigate the process.
You may also get help with contributing in the [dev channel][dev], please join
The [dev channel][dev] is available for any concerns not covered in this guide, please join
us!
[dev]: https://gitter.im/tokio-rs/dev
@@ -153,8 +153,6 @@ The type level example for `tokio_timer::Timeout` provides a good example of a
documentation test:
```
/// # extern crate futures;
/// # extern crate tokio;
/// // import the `timeout` function, usually this is done
/// // with `use tokio::prelude::*`
/// use tokio::prelude::FutureExt;
@@ -192,8 +190,6 @@ If this were a documentation test for the `Timeout::new` function, then the
example would explicitly use `Timeout::new`. For example:
```
/// # extern crate futures;
/// # extern crate tokio;
/// use tokio::timer::Timeout;
/// use futures::Future;
/// use futures::sync::oneshot;
@@ -385,3 +381,63 @@ _Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
## Releasing
Since the Tokio project consists of a number of crates, many of which depend on
each other, releasing new versions to crates.io can involve some complexities.
When releasing a new version of a crate, follow these steps:
1. **Ensure that the release crate has no path dependencies.** When the HEAD
version of a Tokio crate requires unreleased changes in another Tokio crate,
the crates.io dependency on the second crate will be replaced with a path
dependency. Crates with path dependencies cannot be published, so before
publishing the dependent crate, any path dependencies must also be published.
This should be done through a form of depth-first tree traversal:
1. Starting with the first path dependency in the crate to be released,
inspect the `Cargo.toml` for the dependency. If the dependency has any
path dependencies of its own, repeat this step with the first such
dependency.
2. Begin the release process for the path dependency.
3. Once the path dependency has been published to crates.io, update the
dependent crate to depend on the crates.io version.
4. When all path dependencies have been published, the dependent crate may
be published.
To verify that a crate is ready to publish, run:
```bash
bin/publish --dry-run <CRATE NAME> <CRATE VERSION>
```
2. **Update Cargo metadata.** After releasing any path dependencies, update the
`version` field in `Cargo.toml` to the new version, and the `documentation`
field to the docs.rs URL of the new version.
3. **Update other documentation links.** Update the `#![doc(html_root_url)]`
attribute in the crate's `lib.rs` and the "Documentation" link in the crate's
`README.md` to point to the docs.rs URL of the new version.
4. **Update the changelog for the crate.** Each crate in the Tokio repository
has its own `CHANGELOG.md` in that crate's subdirectory. Any changes to that
crate since the last release should be added to the changelog. Change
descriptions may be taken from the Git history, but should be edited to
ensure a consistent format, based on [Keep A Changelog][keep-a-changelog].
Other entries in that crate's changelog may also be used for reference.
5. **Perform a final audit for breaking changes.** Compare the HEAD version of
crate with the Git tag for the most recent release version. If there are any
breaking API changes, determine if those changes can be made without breaking
existing APIs. If so, resolve those issues. Otherwise, if it is necessary to
make a breaking release, update the version numbers to reflect this.
6. **Open a pull request with your changes.** Once that pull request has been
approved by a maintainer and the pull request has been merged, continue to
the next step.
7. **Release the crate.** Run the following command:
```bash
bin/publish <NAME OF CRATE> <VERSION>
```
Your editor and prompt you to edit a message for the tag. Copy the changelog
entry for that release version into your editor and close the window.
[keep-a-changelog]: https://github.com/olivierlacan/keep-a-changelog/blob/master/CHANGELOG.md
+9 -139
View File
@@ -1,144 +1,14 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.15"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.15/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio-async-await",
"tokio-buf",
"tokio-codec",
"tokio-current-thread",
"tokio-executor",
"tokio-fs",
"tokio-io",
"tokio-reactor",
"tokio-signal",
"tokio-sync",
"tokio-threadpool",
"tokio-timer",
"tokio-tcp",
"tokio",
"tokio-macros",
"tokio-test",
"tokio-tls",
"tokio-udp",
"tokio-uds",
"tokio-util",
# Internal
"examples",
"tests-build",
"tests-integration",
]
[features]
default = [
"codec",
"fs",
"io",
"reactor",
"rt-full",
"sync",
"tcp",
"timer",
"udp",
"uds",
]
codec = ["tokio-codec"]
fs = ["tokio-fs"]
io = ["bytes", "tokio-io"]
reactor = ["io", "mio", "tokio-reactor"]
rt-full = [
"num_cpus",
"reactor",
"timer",
"tokio-current-thread",
"tokio-executor",
"tokio-threadpool",
]
sync = ["tokio-sync"]
tcp = ["tokio-tcp"]
timer = ["tokio-timer"]
udp = ["tokio-udp"]
uds = ["tokio-uds"]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
# Only non-optional dependency...
futures = "0.1.20"
# Everything else is optional...
bytes = { version = "0.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
tokio-codec = { version = "0.1.0", path = "tokio-codec", optional = true }
tokio-current-thread = { version = "0.1.3", path = "tokio-current-thread", optional = true }
tokio-fs = { version = "0.1.3", path = "tokio-fs", optional = true }
tokio-io = { version = "0.1.6", path = "tokio-io", optional = true }
tokio-executor = { version = "0.1.5", path = "tokio-executor", optional = true }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor", optional = true }
tokio-sync = { version = "0.1.0", path = "tokio-sync", optional = true }
tokio-threadpool = { version = "0.1.8", path = "tokio-threadpool", optional = true }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp", optional = true }
tokio-udp = { version = "0.1.0", path = "tokio-udp", optional = true }
tokio-timer = { version = "0.2.8", path = "tokio-timer", optional = true }
# Needed until `reactor` is removed from `tokio`.
mio = { version = "0.6.14", optional = true }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds", optional = true }
[dev-dependencies]
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
#tokio = { path = "." }
#tokio-async-await = { path = "./tokio-async-await" }
#tokio-codec = { path = "./tokio-codec" }
#tokio-current-thread = { path = "./tokio-current-thread" }
#tokio-executor = { path = "./tokio-executor" }
#tokio-fs = { path = "./tokio-fs" }
#tokio-io = { path = "./tokio-io" }
#tokio-reactor = { path = "./tokio-reactor" }
#tokio-signal = { path = "./tokio-signal" }
#tokio-tcp = { path = "./tokio-tcp" }
#tokio-threadpool = { path = "./tokio-threadpool" }
#tokio-timer = { path = "./tokio-timer" }
#tokio-tls = { path = "./tokio-tls" }
#tokio-udp = { path = "./tokio-udp" }
#tokio-uds = { path = "./tokio-uds" }
+57 -92
View File
@@ -1,5 +1,7 @@
# Tokio
**NOTE**: Tokio's [`master`](https://github.com/tokio-rs/tokio) is currently undergoing heavy development. This branch and the alpha releases will see API breaking changes. Use the [`v0.1.x`](https://github.com/tokio-rs/tokio/tree/v0.1.x) branch for stable releases.
A runtime for writing reliable, asynchronous, and slim applications with
the Rust programming language. It is:
@@ -14,30 +16,23 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[mit-url]: LICENSE
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio/0.1.15/tokio) |
[Guides](https://tokio.rs/docs/) |
[API Docs](https://docs.rs/tokio/latest/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
## Overview
Tokio is an event-driven, non-blocking I/O platform for writing
@@ -45,63 +40,63 @@ asynchronous applications with the Rust programming language. At a high
level, it provides a few major components:
* A multithreaded, work-stealing based task [scheduler].
* A [reactor] backed by the operating system's event queue (epoll, kqueue,
* A reactor backed by the operating system's event queue (epoll, kqueue,
IOCP, etc...).
* Asynchronous [TCP and UDP][net] sockets.
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/latest/tokio/net/index.html
[scheduler]: https://docs.rs/tokio/latest/tokio/runtime/index.html
## Example
A basic TCP echo server with Tokio:
```rust
extern crate tokio;
use tokio::prelude::*;
use tokio::io::copy;
```rust,no_run
use tokio::net::TcpListener;
use tokio::prelude::*;
fn main() {
// Bind the server's socket.
let addr = "127.0.0.1:12345".parse().unwrap();
let listener = TcpListener::bind(&addr)
.expect("unable to bind TCP listener");
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut listener = TcpListener::bind("127.0.0.1:8080").await?;
// Pull out a stream of sockets for incoming connections
let server = listener.incoming()
.map_err(|e| eprintln!("accept failed = {:?}", e))
.for_each(|sock| {
// Split up the reading and writing parts of the
// socket.
let (reader, writer) = sock.split();
loop {
let (mut socket, _) = listener.accept().await?;
// A future that echos the data and returns how
// many bytes were copied...
let bytes_copied = copy(reader, writer);
tokio::spawn(async move {
let mut buf = [0; 1024];
// ... after which we'll print what happened.
let handle_conn = bytes_copied.map(|amt| {
println!("wrote {:?} bytes", amt)
}).map_err(|err| {
eprintln!("IO error {:?}", err)
});
// In a loop, read data from the socket and write the data back.
loop {
let n = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => return,
Ok(n) => n,
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
return;
}
};
// Spawn the future as a concurrent task.
tokio::spawn(handle_conn)
// Write the data back
if let Err(e) = socket.write_all(&buf[0..n]).await {
eprintln!("failed to write to socket; err = {:?}", e);
return;
}
}
});
// Start the Tokio runtime
tokio::run(server);
}
}
```
More examples can be found [here](examples).
More examples can be found [here](examples). Note that the `master` branch
is currently being updated to use `async` / `await`. The examples are
not fully ported. Examples for stable Tokio can be found
[here](https://github.com/tokio-rs/tokio/tree/v0.1.x/tokio/examples).
## Getting Help
@@ -110,6 +105,8 @@ First, see if the answer to your question can be found in the [Guides] or the
the [Tokio Gitter channel][chat]. We would be happy to try to answer your
question. Last, if that doesn't work, try opening an [issue] with the question.
[Guides]: https://tokio.rs/docs/
[API documentation]: https://docs.rs/tokio/latest/tokio
[chat]: https://gitter.im/tokio-rs/tokio
[issue]: https://github.com/tokio-rs/tokio/issues/new
@@ -121,54 +118,22 @@ project.
[guide]: CONTRIBUTING.md
## Project layout
## Related Projects
The `tokio` crate, found at the root, is primarily intended for use by
application developers. Library authors should depend on the sub crates, which
have greater guarantees of stability.
In addition to the crates in this repository, the Tokio project also maintains
several other libraries, including:
The crates included as part of Tokio are:
* [`tracing`] (formerly `tokio-trace`): A framework for application-level
tracing and async-aware diagnostics.
* [`tokio-async-await`]: Experimental `async` / `await` support.
* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers
`tokio`.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.
* [`tokio-current-thread`]: Schedule the execution of futures on the current
thread.
* [`tokio-executor`]: Task execution related traits and utilities.
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
* [`tokio-tcp`]: TCP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-threadpool`]: Schedules the execution of futures across a pool of
threads.
* [ `tokio-timer`]: Time related APIs.
* [`tokio-udp`]: UDP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-uds`]: Unix Domain Socket bindings for use with `tokio-io` and
`tokio-reactor`.
[`tokio-async-await`]: tokio-async-await
[`tokio-codec`]: tokio-codec
[`tokio-current-thread`]: tokio-current-thread
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-io`]: tokio-io
[`tokio-reactor`]: tokio-reactor
[`tokio-tcp`]: tokio-tcp
[`tokio-threadpool`]: tokio-threadpool
[`tokio-timer`]: tokio-timer
[`tokio-udp`]: tokio-udp
[`tokio-uds`]: tokio-uds
[`tracing`]: https://github.com/tokio-rs/tracing
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Supported Rust Versions
+108
View File
@@ -0,0 +1,108 @@
trigger: ["master"]
pr: ["master"]
variables:
RUSTFLAGS: -Dwarnings
nightly: nightly-2019-11-16
jobs:
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
rust: stable
displayName: Test tokio
cross: true
crates:
- tokio
- tests-integration
# Test sub crates
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
rust: stable
crates:
- tokio-macros
- tokio-test
- tokio-tls
- tokio-util
- examples
# Run tests from `tests-build`. This requires a different process
- template: ci/azure-test-build.yml
parameters:
name: test_build
displayName: Test build permutations
rust: stable
# Run loom tests
- template: ci/azure-loom.yml
parameters:
name: loom
rust: stable
crates:
- tokio
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross
rust: stable
# Check each feature works properly
- template: ci/azure-check-features.yml
parameters:
rust: $(nightly)
name: check_features
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust: 1.39.0
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
rust: stable
name: rustfmt
# Apply clippy lints to all crates
- template: ci/azure-clippy.yml
parameters:
rust: stable
name: clippy
# Check doc generation
- template: ci/azure-check-docs.yml
parameters:
rust: $(nightly)
name: docs
# - template: ci/azure-tsan.yml
# parameters:
# name: tsan
# rust: stable
- template: ci/azure-deploy-docs.yml
parameters:
rust: stable
dependsOn:
- rustfmt
- clippy
- test_tokio
- test_linux
- test_build
- loom
- cross
- minrust
- check_features
# - tsan
Executable
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
set -e
USAGE="Publish a new release of a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-v, --verbose Use verbose Cargo output
-d, --dry-run Perform a dry run (do not publish or tag the release)
-h, --help Show this help text and exit"
DRY_RUN=""
VERBOSE=""
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
verify() {
status "Verifying" "if $CRATE v$VERSION can be released"
ACTUAL=$(cargo pkgid | sed -n 's/.*#\(.*\)/\1/p')
if [ "$ACTUAL" != "$VERSION" ]; then
err "expected to release version $VERSION, but Cargo.toml contained $ACTUAL"
exit 1
fi
if git tag -l | grep -Fxq "$TAG" ; then
err "git tag \`$TAG\` already exists"
exit 1
fi
PATH_DEPS=$(grep -F "path = \"" Cargo.toml | sed -e 's/^/ /')
if [ -n "$PATH_DEPS" ]; then
err "crate \`$CRATE\` contained path dependencies:\n$PATH_DEPS"
echo "path dependencies must be removed prior to release"
exit 1
fi
}
release() {
status "Releasing" "$CRATE v$VERSION"
cargo package $VERBOSE
cargo publish $VERBOSE $DRY_RUN
status "Tagging" "$TAG"
if [ -n "$DRY_RUN" ]; then
echo "# git tag $TAG && git push --tags"
else
git tag "$TAG" && git push --tags
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-v|--verbose)
VERBOSE="--verbose"
set +x
shift
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
(cd "$CRATE" && verify && release )
else
err "no such crate \"$CRATE\""
exit 1
fi
Executable
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
set -e
USAGE="Update links to docs.rs in a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-d, --dry-run Perform a dry run (do not modify any file)
-h, --help Show this help text and exit"
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
c1grep() { grep "$@" || test $? = 1; }
update_versions_in_doc() {
# Print what is being/would be done
if [ -n "$DRY_RUN" ]; then
local MSG="Would change:"
else
local MSG="Updating:"
fi
git grep -lr "docs.rs/$CRATE/" \
| xargs sed --quiet \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|gp" \
| sed -e "s/^/$MSG /"
# Apply changes if not in dry run
if [ -z "$DRY_RUN" ]; then
git grep -lr "docs.rs/$CRATE/" \
| xargs sed -i \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|g"
fi
}
update() {
update_versions_in_doc
}
show_outdated() {
OUTDATED=$(git grep -rn "docs.rs/$CRATE/" \
| c1grep -v "$VERSION" \
| sed -e 's/^/ - /')
if [[ -n "$OUTDATED" ]]; then
echo "Found the following links to docs.rs with an outdated version:"
echo "$OUTDATED"
echo
else
echo "Nothing to do."
exit 1
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
# Does not cd in order to update everywhere
show_outdated && update
else
err "no such crate \"$CRATE\""
exit 1
fi
+29
View File
@@ -0,0 +1,29 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+15
View File
@@ -0,0 +1,15 @@
jobs:
# Check docs
- job: ${{ parameters.name }}
displayName: Check docs
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
RUSTDOCFLAGS="--cfg docsrs" cargo doc --lib --no-deps --all-features
displayName: Check docs
+32
View File
@@ -0,0 +1,32 @@
jobs:
- job: ${{ parameters.name }}
displayName: Check features
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo install cargo-hack
displayName: Install cargo-hack
# Check each feature works properly
# * --each-feature
# run for each feature which includes --no-default-features and default features of package
# * -Z avoid-dev-deps
# build without dev-dependencies to avoid https://github.com/rust-lang/cargo/issues/4866
# tracking-issue: https://github.com/rust-lang/cargo/issues/5133
- script: cargo hack check --all --each-feature -Z avoid-dev-deps
displayName: cargo hack check --all --each-feature
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+16
View File
@@ -0,0 +1,16 @@
jobs:
- job: ${{ parameters.name }}
displayName: Clippy
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add clippy
cargo clippy --version
displayName: Install clippy
- script: |
cargo clippy --all --all-features -- -A clippy::mutex-atomic
displayName: cargo clippy --all
+44
View File
@@ -0,0 +1,44 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
i686:
vmImage: ubuntu-16.04
target: i686-unknown-linux-gnu
powerpc:
vmImage: ubuntu-16.04
target: powerpc-unknown-linux-gnu
powerpc64:
vmImage: ubuntu-16.04
target: powerpc64-unknown-linux-gnu
mips:
vmImage: ubuntu-16.04
target: mips-unknown-linux-gnu
arm:
vmImage: ubuntu-16.04
target: arm-linux-androideabi
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: sudo apt-get update
displayName: apt-get update
- script: sudo apt-get install gcc-multilib
displayName: Install gcc-multilib
- script: cargo install cross
displayName: Install cross
# Always patch
- template: azure-patch-crates.yml
- script: cross check --all --exclude tokio-tls --target $(target)
displayName: Check source
# - script: cross check --tests --all --exclude tokio-tls --target $(target)
# displayName: Check tests
+39
View File
@@ -0,0 +1,39 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
# rust_version: stable
rust_version: ${{ parameters.rust }}
- script: |
cargo doc --all --no-deps --all-features
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+33
View File
@@ -0,0 +1,33 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain none
export PATH=$PATH:$HOME/.cargo/bin
rustup toolchain install $RUSTUP_TOOLCHAIN
rustup default $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --profile minimal --default-toolchain none
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
rustup toolchain install %RUSTUP_TOOLCHAIN%
rustup default %RUSTUP_TOOLCHAIN%
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustup toolchain list
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%B' | grep -qF '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+18
View File
@@ -0,0 +1,18 @@
jobs:
- job: ${{ parameters.name }}
displayName: Loom tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- ${{ each crate in parameters.crates }}:
- script: RUSTFLAGS="--cfg loom" cargo test --lib --release -- --test-threads=1 --nocapture
env:
LOOM_MAX_PREEMPTIONS: 1
CI: 'True'
displayName: test ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+17
View File
@@ -0,0 +1,17 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add rustfmt
cargo fmt --version
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+17
View File
@@ -0,0 +1,17 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: 'Ubuntu 16.04'
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: cargo install cargo-hack
displayName: Install cargo-hack
- script: cargo hack test --each-feature
displayName: cargo hack test --each-feature
workingDirectory: $(Build.SourcesDirectory)/tests-build
+19
View File
@@ -0,0 +1,19 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
# Check benches
- script: cargo check --benches --all
displayName: Check benchmarks
+42
View File
@@ -0,0 +1,42 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
# Run with all crate features
- script: cargo test --all-features
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate }} - cargo test --all-features
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
# Run with all crate features
- script: cargo test --all-features
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate }} - cargo test --all-features
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+34
View File
@@ -0,0 +1,34 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+8
View File
@@ -0,0 +1,8 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-macros = { path = "tokio-macros" }
tokio-test = { path = "tokio-test" }
tokio-tls = { path = "tokio-tls" }
tokio-util = { path = "tokio-util" }
+2
View File
@@ -8,6 +8,8 @@ race:Weak*drop
# `std` mpsc is not used in any Tokio code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
race:std*lang_start
race:drop*std::thread*
# Probably more fences in std.
race:__call_tls_dtors
+52
View File
@@ -0,0 +1,52 @@
[package]
name = "examples"
version = "0.0.0"
publish = false
edition = "2018"
[dev-dependencies]
tokio = { version = "0.2.0", path = "../tokio", features = ["full"] }
tokio-util = { version = "0.2.0", path = "../tokio-util", features = ["full"] }
bytes = "0.5.0"
futures = "0.3.0"
[[example]]
name = "chat"
path = "chat.rs"
[[example]]
name = "connect"
path = "connect.rs"
[[example]]
name = "echo-udp"
path = "echo-udp.rs"
[[example]]
name = "echo"
path = "echo.rs"
[[example]]
name = "hello_world"
path = "hello_world.rs"
[[example]]
name = "print_each_packet"
path = "print_each_packet.rs"
[[example]]
name = "proxy"
path = "proxy.rs"
[[example]]
name = "tinydb"
path = "tinydb.rs"
[[example]]
name = "udp-client"
path = "udp-client.rs"
[[example]]
name = "udp-codec"
path = "udp-codec.rs"
+4 -58
View File
@@ -1,60 +1,6 @@
## Examples of how to use Tokio
This directory contains a number of examples showcasing various capabilities of
the `tokio` crate.
All examples can be executed with:
```
cargo run --example $name
```
A high level description of each example is:
* [`hello_world`](hello_world.rs) - a tiny server that writes "hello world" to
all connected clients and then terminates the connection, should help see how
to create and initialize `tokio`.
* [`echo`](echo.rs) - this is your standard TCP "echo server" which accepts
connections and then echos back any contents that are read from each connected
client.
* [`print_each_packet`](print_each_packet.rs) - this server will create a TCP
listener, accept connections in a loop, and put down in the stdout everything
that's read off of each TCP connection.
* [`echo-udp`](echo-udp.rs) - again your standard "echo server", except for UDP
instead of TCP. This will echo back any packets received to the original
sender.
* [`connect`](connect.rs) - this is a `nc`-like clone which can be used to
interact with most other examples. The program creates a TCP connection or UDP
socket to sends all information read on stdin to the remote peer, displaying
any data received on stdout. Often quite useful when interacting with the
various other servers here!
* [`chat`](chat.rs) - this spins up a local TCP server which will broadcast from
any connected client to all other connected clients. You can connect to this
in multiple terminals and use it to chat between the terminals.
* [`chat-combinator`](chat-combinator.rs) - Similar to `chat`, but this uses a
much more functional programming approach using combinators.
* [`proxy`](proxy.rs) - an example proxy server that will forward all connected
TCP clients to the remote address specified when starting the program.
* [`tinyhttp`](tinyhttp.rs) - a tiny HTTP/1.1 server which doesn't support HTTP
request bodies showcasing running on multiple cores, working with futures and
spawning tasks, and finally framing a TCP connection to discrete
request/response objects.
* [`tinydb`](tinydb.rs) - an in-memory database which shows sharing state
between all connected clients, notably the key/value store of this database.
* [`udp-client`](udp-client.rs) - a simple `send_dgram`/`recv_dgram` example.
* [`manual-runtime`](manual-runtime.rs) - manually composing a runtime.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
The `master` branch is currently being updated to use `async` / `await`.
The examples are not fully ported. Examples for stable Tokio can be
found
[here](https://github.com/tokio-rs/tokio/tree/v0.1.x/tokio/examples).
-167
View File
@@ -1,167 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! Because we are here running the reactor/executor on the same thread instead
//! of a threadpool, we can avoid full synchronization with Arc + Mutex and use
//! Rc + RefCell instead. The max performance is however limited to a CPU HW
//! thread.
//!
//! You can test this out by running:
//!
//! cargo run --example chat-combinator-current-thread
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
extern crate futures;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::runtime::current_thread::{Runtime, TaskExecutor};
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::rc::Rc;
use std::cell::RefCell;
fn main() -> Result<(), Box<std::error::Error>> {
let mut runtime = Runtime::new().unwrap();
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio current_thread runtime, so it will be single-
// threaded. The `Rc<RefCell<...>>` allows state to be shared across the tasks.
let connections = Rc::new(RefCell::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| {println!("failed to accept socket; error = {:?}", e); e})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
let mut conns = connections.borrow_mut();
conns.insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.borrow_mut();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn locally a task to process the connection
TaskExecutor::current().spawn_local(Box::new(connection.then(move |_| {
let mut conns = connections.borrow_mut();
conns.remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}))).unwrap();
Ok(())
})
.map_err(|err| println!("error occurred: {:?}", err));
// Spawn srv itself
runtime.spawn(srv);
// Execute server
runtime.run().unwrap();
Ok(())
}
-152
View File
@@ -1,152 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
extern crate futures;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::sync::{Arc, Mutex};
fn main() -> Result<(), Box<std::error::Error>> {
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio runtime, so it will be multi-threaded. The
// `Arc<Mutex<...>>` allows state to be shared across the threads.
let connections = Arc::new(Mutex::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| {println!("failed to accept socket; error = {:?}", e); e})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
connections.lock().unwrap().insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.lock().unwrap();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn a task to process the connection
tokio::spawn(connection.then(move |_| {
connections.lock().unwrap().remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}));
Ok(())
})
.map_err(|err| println!("error occurred: {:?}", err));
// execute server
tokio::run(srv);
Ok(())
}
+154 -374
View File
@@ -24,29 +24,61 @@
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
#[macro_use]
extern crate futures;
extern crate bytes;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use tokio::sync::{mpsc, Mutex};
use tokio_util::codec::{Framed, LinesCodec, LinesCodecError};
use futures::{SinkExt, Stream, StreamExt};
use std::collections::HashMap;
use std::env;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use std::pin::Pin;
use std::sync::Arc;
use std::task::{Context, Poll};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = env::args().nth(1).unwrap_or("127.0.0.1:6142".to_string());
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let mut listener = TcpListener::bind(&addr).await?;
println!("server running on {}", addr);
loop {
// Asynchronously wait for an inbound TcpStream.
let (stream, addr) = listener.accept().await?;
// Clone a handle to the `Shared` state for the new connection.
let state = Arc::clone(&state);
// Spawn our handler to be run asynchronously.
tokio::spawn(async move {
if let Err(e) = process(state, stream, addr).await {
println!("an error occured; error = {:?}", e);
}
});
}
}
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<Bytes>;
type Tx = mpsc::UnboundedSender<String>;
/// Shorthand for the receive half of the message channel.
type Rx = mpsc::UnboundedReceiver<Bytes>;
type Rx = mpsc::UnboundedReceiver<String>;
/// Data that is shared between all peers in the chat server.
///
@@ -60,64 +92,18 @@ struct Shared {
/// The state for each connected client.
struct Peer {
/// Name of the peer.
///
/// When a client connects, the first line sent is treated as the client's
/// name (like alice or bob). The name is used to preface all messages that
/// arrive from the client so that we can simulate a real chat server:
///
/// ```text
/// alice: Hello everyone.
/// bob: Welcome to telnet chat!
/// ```
name: BytesMut,
/// The TCP socket wrapped with the `Lines` codec, defined below.
///
/// This handles sending and receiving data on the socket. When using
/// `Lines`, we can work at the line level instead of having to manage the
/// raw byte operations.
lines: Lines,
/// Handle to the shared chat state.
///
/// This is used to broadcast messages read off the socket to all connected
/// peers.
state: Arc<Mutex<Shared>>,
lines: Framed<TcpStream, LinesCodec>,
/// Receive half of the message channel.
///
/// This is used to receive messages from peers. When a message is received
/// off of this `Rx`, it will be written to the socket.
rx: Rx,
/// Client socket address.
///
/// The socket address is used as the key in the `peers` HashMap. The
/// address is saved so that the `Peer` drop implementation can clean up its
/// entry.
addr: SocketAddr,
}
/// Line based codec
///
/// This decorates a socket and presents a line based read / write interface.
///
/// As a user of `Lines`, we can focus on working at the line level. So, we send
/// and receive values that represent entire lines. The `Lines` codec will
/// handle the encoding and decoding as well as reading from and writing to the
/// socket.
#[derive(Debug)]
struct Lines {
/// The TCP socket.
socket: TcpStream,
/// Buffer used when reading from the socket. Data is not returned from this
/// buffer until an entire line has been read.
rd: BytesMut,
/// Buffer used to stage data before writing it to the socket.
wr: BytesMut,
}
impl Shared {
@@ -127,349 +113,143 @@ impl Shared {
peers: HashMap::new(),
}
}
/// Send a `LineCodec` encoded message to every peer, except
/// for the sender.
async fn broadcast(&mut self, sender: SocketAddr, message: &str) {
for peer in self.peers.iter_mut() {
if *peer.0 != sender {
let _ = peer.1.send(message.into());
}
}
}
}
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
async fn new(
state: Arc<Mutex<Shared>>,
lines: Framed<TcpStream, LinesCodec>,
) -> io::Result<Peer> {
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
let addr = lines.get_ref().peer_addr()?;
// Create a channel for this peer
let (tx, rx) = mpsc::unbounded();
let (tx, rx) = mpsc::unbounded_channel();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
state.lock().await.peers.insert(addr, tx);
Peer {
name,
lines,
state,
rx,
addr,
}
Ok(Peer { lines, rx })
}
}
/// This is where a connected client is managed.
///
/// A `Peer` is also a future representing completely processing the client.
///
/// When a `Peer` is created, the first line (representing the client's name)
/// has already been read. When the socket closes, the `Peer` future completes.
///
/// While processing, the peer future implementation will:
///
/// 1) Receive messages on its message channel and write them to the socket.
/// 2) Receive messages from the socket and broadcast them to all peers.
///
impl Future for Peer {
type Item = ();
type Error = io::Error;
#[derive(Debug)]
enum Message {
/// A message that should be broadcasted to others.
Broadcast(String),
fn poll(&mut self) -> Poll<(), io::Error> {
// Tokio (and futures) use cooperative scheduling without any
// preemption. If a task never yields execution back to the executor,
// then other tasks may be starved.
//
// To deal with this, robust applications should not have any unbounded
// loops. In this example, we will read at most `LINES_PER_TICK` lines
// from the client on each tick.
//
// If the limit is hit, the current task is notified, informing the
// executor to schedule the task again asap.
const LINES_PER_TICK: usize = 10;
// Receive all messages from peers.
for i in 0..LINES_PER_TICK {
// Polling an `UnboundedReceiver` cannot fail, so `unwrap` here is
// safe.
match self.rx.poll().unwrap() {
Async::Ready(Some(v)) => {
// Buffer the line. Once all lines are buffered, they will
// be flushed to the socket (right below).
self.lines.buffer(&v);
// If this is the last iteration, the loop will break even
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
task::current().notify();
}
}
_ => break,
}
}
// Flush the write buffer to the socket
let _ = self.lines.poll_flush()?;
// Read new lines from the socket
while let Async::Ready(line) = self.lines.poll()? {
println!("Received line ({:?}) : {:?}", self.name, line);
if let Some(message) = line {
// Append the peer's name to the front of the line:
let mut line = self.name.clone();
line.extend_from_slice(b": ");
line.extend_from_slice(&message);
line.extend_from_slice(b"\r\n");
// We're using `Bytes`, which allows zero-copy clones (by
// storing the data in an Arc internally).
//
// However, before cloning, we must freeze the data. This
// converts it from mutable -> immutable, allowing zero copy
// cloning.
let line = line.freeze();
// Now, send the line to all other peers
for (addr, tx) in &self.state.lock().unwrap().peers {
// Don't send the message to ourselves
if *addr != self.addr {
// The send only fails if the rx half has been dropped,
// however this is impossible as the `tx` half will be
// removed from the map before the `rx` is dropped.
tx.unbounded_send(line.clone()).unwrap();
}
}
} else {
// EOF was reached. The remote client has disconnected. There is
// nothing more to do.
return Ok(Async::Ready(()));
}
}
// As always, it is important to not just return `NotReady` without
// ensuring an inner future also returned `NotReady`.
//
// We know we got a `NotReady` from either `self.rx` or `self.lines`, so
// the contract is respected.
Ok(Async::NotReady)
}
/// A message that should be received by a client
Received(String),
}
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
}
}
// Peer implements `Stream` in a way that polls both the `Rx`, and `Framed` types.
// A message is produced whenever an event is ready until the `Framed` stream returns `None`.
impl Stream for Peer {
type Item = Result<Message, LinesCodecError>;
impl Lines {
/// Create a new `Lines` codec backed by the socket
fn new(socket: TcpStream) -> Self {
Lines {
socket,
rd: BytesMut::new(),
wr: BytesMut::new(),
}
}
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// First poll the `UnboundedReceiver`.
/// Buffer a line.
///
/// This writes the line to an internal buffer. Calls to `poll_flush` will
/// attempt to flush this buffer to the socket.
fn buffer(&mut self, line: &[u8]) {
// Ensure the buffer has capacity. Ideally this would not be unbounded,
// but to keep the example simple, we will not limit this.
self.wr.reserve(line.len());
// Push the line onto the end of the write buffer.
//
// The `put` function is from the `BufMut` trait.
self.wr.put(line);
}
/// Flush the write buffer to the socket
fn poll_flush(&mut self) -> Poll<(), io::Error> {
// As long as there is buffered data to write, try to write it.
while !self.wr.is_empty() {
// Try to write some bytes to the socket
let n = try_ready!(self.socket.poll_write(&self.wr));
// As long as the wr is not empty, a successful write should
// never write 0 bytes.
assert!(n > 0);
// This discards the first `n` bytes of the buffer.
let _ = self.wr.split_to(n);
if let Poll::Ready(Some(v)) = self.rx.poll_next_unpin(cx) {
return Poll::Ready(Some(Ok(Message::Received(v))));
}
Ok(Async::Ready(()))
}
// Secondly poll the `Framed` stream.
let result: Option<_> = futures::ready!(self.lines.poll_next_unpin(cx));
/// Read data from the socket.
///
/// This only returns `Ready` when the socket has closed.
fn fill_read_buf(&mut self) -> Poll<(), io::Error> {
loop {
// Ensure the read buffer has capacity.
//
// This might result in an internal allocation.
self.rd.reserve(1024);
Poll::Ready(match result {
// We've received a message we should broadcast to others.
Some(Ok(message)) => Some(Ok(Message::Broadcast(message))),
// Read data into the buffer.
let n = try_ready!(self.socket.read_buf(&mut self.rd));
// An error occured.
Some(Err(e)) => Some(Err(e)),
if n == 0 {
return Ok(Async::Ready(()));
}
}
}
}
impl Stream for Lines {
type Item = BytesMut;
type Error = io::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
// First, read any new data that might have been received off the socket
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
if let Some(pos) = pos {
// Remove the line from the read buffer and set it to `line`.
let mut line = self.rd.split_to(pos + 2);
// Drop the trailing \r\n
line.split_off(pos);
// Return the line
return Ok(Async::Ready(Some(line)));
}
if sock_closed {
Ok(Async::Ready(None))
} else {
Ok(Async::NotReady)
}
}
}
/// Spawn a task to manage the socket.
///
/// This will read the first line from the socket to identify the client, then
/// add the client to the set of connected peers in the chat service.
fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// Wrap the socket with the `Lines` codec that we wrote above.
//
// By doing this, we can operate at the line level instead of doing raw byte
// manipulation.
let lines = Lines::new(socket);
// The first line is treated as the client's name. The client is not added
// to the set of connected peers until this line is received.
//
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
// Process the first received line as the client's name.
.and_then(|(name, lines)| {
// If `name` is `None`, then the client disconnected without
// actually sending a line of data.
//
// Since the connection is closed, there is no further work that we
// need to do. So, we just terminate processing by returning
// `future::ok()`.
//
// The problem is that only a single future type can be returned
// from a combinator closure, but we want to return both
// `future::ok()` and `Peer` (below).
//
// This is a common problem, so the `futures` crate solves this by
// providing the `Either` helper enum that allows creating a single
// return type that covers two concrete future types.
let name = match name {
Some(name) => name,
None => {
// The remote client closed the connection without sending
// any data.
return Either::A(future::ok(()));
}
};
println!("`{:?}` is joining the chat", name);
// Create the peer.
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
// The stream has been exhausted.
None => None,
})
// Task futures have an error of type `()`, this ensures we handle the
// error. We do this by printing the error to STDOUT.
.map_err(|e| {
println!("connection error = {:?}", e);
});
// Spawn the task. Internally, this submits the task to a thread pool.
tokio::spawn(connection);
}
}
pub fn main() -> Result<(), Box<std::error::Error>> {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
/// Process an individual chat client
async fn process(
state: Arc<Mutex<Shared>>,
stream: TcpStream,
addr: SocketAddr,
) -> Result<(), Box<dyn Error>> {
let mut lines = Framed::new(stream, LinesCodec::new());
let addr = "127.0.0.1:6142".parse()?;
// Send a prompt to the client to enter their username.
lines
.send(String::from("Please enter your username:"))
.await?;
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr)?;
// Read the first line from the `LineCodec` stream to get the username.
let username = match lines.next().await {
Some(Ok(line)) => line,
// We didn't get a line so we return early here.
_ => {
println!("Failed to get username from {}. Client disconnected.", addr);
return Ok(());
}
};
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
// Register our peer with state which internally sets up some channels.
let mut peer = Peer::new(state.clone(), lines).await?;
println!("server running on localhost:6142");
// A client has connected, let's let everyone know.
{
let mut state = state.lock().await;
let msg = format!("{} has joined the chat", username);
println!("{}", msg);
state.broadcast(addr, &msg).await;
}
// Process incoming messages until our stream is exhausted by a disconnect.
while let Some(result) = peer.next().await {
match result {
// A message was received from the current user, we should
// broadcast this message to the other users.
Ok(Message::Broadcast(msg)) => {
let mut state = state.lock().await;
let msg = format!("{}: {}", username, msg);
state.broadcast(addr, &msg).await;
}
// A message was received from a peer. Send it to the
// current user.
Ok(Message::Received(msg)) => {
peer.lines.send(msg).await?;
}
Err(e) => {
println!(
"an error occured while processing messages for {}; error = {:?}",
username, e
);
}
}
}
// If this section is reached it means that the client was disconnected!
// Let's let everyone still connected know about it.
{
let mut state = state.lock().await;
state.peers.remove(&addr);
let msg = format!("{} has left the chat", username);
println!("{}", msg);
state.broadcast(addr, &msg).await;
}
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
Ok(())
}
+134 -175
View File
@@ -14,22 +14,30 @@
//! this repository! Many of them recommend running this as a simple "hook up
//! stdin/stdout to a server" to get up and running.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use tokio::io;
use tokio::sync::{mpsc, oneshot};
use tokio_util::codec::{FramedRead, FramedWrite};
use futures::{Stream, StreamExt};
use std::env;
use std::io::{self, Read, Write};
use std::error::Error;
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let (tx, rx) = oneshot::channel();
tokio::spawn(async move {
run().await.unwrap();
tx.send(()).unwrap();
});
fn main() -> Result<(), Box<std::error::Error>> {
rx.await.map_err(Into::into)
}
// Currently, we need to spawn the initial future due to https://github.com/tokio-rs/tokio/issues/1356
async fn run() -> Result<(), Box<dyn Error>> {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
@@ -47,44 +55,127 @@ fn main() -> Result<(), Box<std::error::Error>> {
};
let addr = addr.parse::<SocketAddr>()?;
// Right now Tokio doesn't support a handle to stdin running on the event
// loop, so we farm out that work to a separate thread. This thread will
// read data (with blocking I/O) from stdin and then send it to the event
// loop over a standard futures channel.
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(|| read_stdin(stdin_tx));
let stdin_rx = stdin_rx.map_err(|_| panic!("errors not possible on rx"));
let stdin = stdin();
let stdout = FramedWrite::new(io::stdout(), codec::Bytes);
// Now that we've got our stdin read we either set up our TCP connection or
// our UDP connection to get a stream of bytes we're going to emit to
// stdout.
let stdout = if tcp {
tcp::connect(&addr, Box::new(stdin_rx))?
if tcp {
tcp::connect(&addr, stdin, stdout).await?;
} else {
udp::connect(&addr, Box::new(stdin_rx))?
};
udp::connect(&addr, stdin, stdout).await?;
}
// And now with our stream of bytes to write to stdout, we execute that in
// the event loop! Note that this is doing blocking I/O to emit data to
// stdout, and in general it's a no-no to do that sort of work on the event
// loop. In this case, though, we know it's ok as the event loop isn't
// otherwise running anything useful.
let mut out = io::stdout();
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
})
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
Ok(())
}
mod codec {
// Temporary work around for stdin blocking the stream
fn stdin() -> impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin {
let mut stdin = FramedRead::new(io::stdin(), codec::Bytes);
let (tx, rx) = mpsc::unbounded_channel();
tokio::spawn(async move {
while let Some(res) = stdin.next().await {
let _ = tx.send(res);
}
});
rx
}
mod tcp {
use super::codec;
use futures::{future, Sink, SinkExt, Stream, StreamExt};
use std::{error::Error, io, net::SocketAddr};
use tokio::net::TcpStream;
use tokio_util::codec::{FramedRead, FramedWrite};
pub async fn connect(
addr: &SocketAddr,
stdin: impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin,
mut stdout: impl Sink<Vec<u8>, Error = io::Error> + Unpin,
) -> Result<(), Box<dyn Error>> {
let mut stream = TcpStream::connect(addr).await?;
let (r, w) = stream.split();
let sink = FramedWrite::new(w, codec::Bytes);
let mut stream = FramedRead::new(r, codec::Bytes)
.filter_map(|i| match i {
Ok(i) => future::ready(Some(i)),
Err(e) => {
println!("failed to read from socket; error={}", e);
future::ready(None)
}
})
.map(Ok);
match future::join(stdin.forward(sink), stdout.send_all(&mut stream)).await {
(Err(e), _) | (_, Err(e)) => Err(e.into()),
_ => Ok(()),
}
}
}
mod udp {
use tokio::net::udp::{RecvHalf, SendHalf};
use tokio::net::UdpSocket;
use futures::{future, Sink, SinkExt, Stream, StreamExt};
use std::error::Error;
use std::io;
use std::net::SocketAddr;
pub async fn connect(
addr: &SocketAddr,
stdin: impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin,
stdout: impl Sink<Vec<u8>, Error = io::Error> + Unpin,
) -> Result<(), Box<dyn Error>> {
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let bind_addr = if addr.ip().is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
};
let socket = UdpSocket::bind(&bind_addr).await?;
socket.connect(addr).await?;
let (mut r, mut w) = socket.split();
future::try_join(send(stdin, &mut w), recv(stdout, &mut r)).await?;
Ok(())
}
async fn send(
mut stdin: impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin,
writer: &mut SendHalf,
) -> Result<(), io::Error> {
while let Some(item) = stdin.next().await {
let buf = item?;
writer.send(&buf[..]).await?;
}
Ok(())
}
async fn recv(
mut stdout: impl Sink<Vec<u8>, Error = io::Error> + Unpin,
reader: &mut RecvHalf,
) -> Result<(), io::Error> {
loop {
let mut buf = vec![0; 1024];
let n = reader.recv(&mut buf[..]).await?;
if n > 0 {
stdout.send(buf).await?;
}
}
}
}
mod codec {
use bytes::{BufMut, BytesMut};
use tokio::codec::{Encoder, Decoder};
use std::io;
use tokio_util::codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
///
@@ -96,13 +187,13 @@ mod codec {
pub struct Bytes;
impl Decoder for Bytes {
type Item = BytesMut;
type Item = Vec<u8>;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<Vec<u8>>> {
if buf.len() > 0 {
let len = buf.len();
Ok(Some(buf.split_to(len)))
Ok(Some(buf.split_to(len).into_iter().collect()))
} else {
Ok(None)
}
@@ -119,135 +210,3 @@ mod codec {
}
}
}
mod tcp {
use tokio;
use tokio::net::TcpStream;
use tokio::prelude::*;
use tokio::codec::Decoder;
use bytes::BytesMut;
use codec::Bytes;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>>
{
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
// to start forwarding data.
//
// First we use the `Io::framed` method with a simple implementation of
// a `Codec` (listed below) that just ships bytes around. We then split
// that in two to work with the stream and sink separately.
//
// Half of the work we're going to do is to take all data we receive on
// `stdin` and send that along the TCP stream (`sink`). The second half
// is to take all the data we receive (`stream`) and then write that to
// stdout. We'll be passing this handle back out from this method.
//
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
let stream = Box::new(tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream());
Ok(stream)
}
}
mod udp {
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>>
{
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
"0.0.0.0:0".parse()?
} else {
"[::]:0".parse()?
};
let udp = match UdpSocket::bind(&addr_to_bind) {
Ok(udp) => udp,
Err(_) => Err("failed to bind socket")?,
};
// Like above with TCP we use an instance of `Bytes` codec to transform
// this UDP socket into a framed sink/stream which operates over
// discrete values. In this case we're working with *pairs* of socket
// addresses and byte buffers.
let (sink, stream) = UdpFramed::new(udp, Bytes).split();
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
let forward_stdin = stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
let receive = stream.filter_map(move |(chunk, src)| {
if src == addr {
Some(chunk.into())
} else {
None
}
});
let stream = Box::new(future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
}).flatten_stream());
Ok(stream)
}
}
// Our helper method which will read data from stdin and send it along the
// sender provided.
fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
let mut stdin = io::stdin();
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
tx = match tx.send(buf).wait() {
Ok(tx) => tx,
Err(_) => break,
};
}
}
+21 -26
View File
@@ -10,16 +10,12 @@
//!
//! Each line you type in to the `nc` terminal should be echo'd back to you!
#![deny(warnings)]
#![warn(rust_2018_idioms)]
#[macro_use]
extern crate futures;
extern crate tokio;
use std::{env, io};
use std::error::Error;
use std::net::SocketAddr;
use tokio::prelude::*;
use std::{env, io};
use tokio;
use tokio::net::UdpSocket;
struct Server {
@@ -28,47 +24,46 @@ struct Server {
to_send: Option<(usize, SocketAddr)>,
}
impl Future for Server {
type Item = ();
type Error = io::Error;
impl Server {
async fn run(self) -> Result<(), io::Error> {
let Server {
mut socket,
mut buf,
mut to_send,
} = self;
fn poll(&mut self) -> Poll<(), io::Error> {
loop {
// First we check to see if there's a message we need to echo back.
// If so then we try to send it back to the original source, waiting
// until it's writable and we're able to do so.
if let Some((size, peer)) = self.to_send {
let amt = try_ready!(self.socket.poll_send_to(&self.buf[..size], &peer));
if let Some((size, peer)) = to_send {
let amt = socket.send_to(&buf[..size], &peer).await?;
println!("Echoed {}/{} bytes to {}", amt, size, peer);
self.to_send = None;
}
// If we're here then `to_send` is `None`, so we take a look for the
// next message we're going to echo back.
self.to_send = Some(try_ready!(self.socket.poll_recv_from(&mut self.buf)));
to_send = Some(socket.recv_from(&mut buf).await?);
}
}
}
fn main() -> Result<(), Box<std::error::Error>> {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
let socket = UdpSocket::bind(&addr)?;
let socket = UdpSocket::bind(&addr).await?;
println!("Listening on: {}", socket.local_addr()?);
let server = Server {
socket: socket,
socket,
buf: vec![0; 1024],
to_send: None,
};
// This starts the server task.
//
// `map_err` handles the error by logging it and maps the future to a type
// that can be spawned.
//
// `tokio::run` spawns the task on the Tokio runtime and starts running.
tokio::run(server.map_err(|e| println!("server error = {:?}", e)));
server.run().await?;
Ok(())
}
+37 -75
View File
@@ -19,97 +19,59 @@
//! you! If you open up multiple terminals running the `connect` example you
//! should be able to see them all make progress simultaneously.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use tokio::io;
use tokio;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
use std::error::Error;
fn main() -> Result<(), Box<std::error::Error>> {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr)?;
// above and must be associated with an event loop.
let mut listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We just want to copy all data read from the socket back onto the
// socket itself (e.g. "echo"). We can use the standard `io::copy`
// combinator in the `tokio-core` crate to do precisely this!
//
// The `copy` function takes two arguments, where to read from and where
// to write to. We only have one argument, though, with `socket`.
// Luckily there's a method, `Io::split`, which will split an Read/Write
// stream into its two halves. This operation allows us to work with
// each stream independently, such as pass them as two arguments to the
// `copy` function.
//
// The `copy` function then returns a future, and this future will be
// resolved when the copying operation is complete, resolving to the
// amount of data that was copied.
let (reader, writer) = socket.split();
let amt = io::copy(reader, writer);
loop {
// Asynchronously wait for an inbound socket.
let (mut socket, _) = listener.accept().await?;
// After our copy operation is complete we just print out some helpful
// information.
let msg = amt.then(move |result| {
match result {
Ok((amt, _, _)) => println!("wrote {} bytes", amt),
Err(e) => println!("error: {}", e),
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(async move {
let mut buf = [0; 1024];
// In a loop, read data from the socket and write the data back.
loop {
let n = socket
.read(&mut buf)
.await
.expect("failed to read data from socket");
if n == 0 {
return;
}
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(msg)
socket
.write_all(&buf[0..n])
.await
.expect("failed to write data to socket");
}
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
}
+9 -33
View File
@@ -11,47 +11,23 @@
//!
//! cargo run --example hello_world
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use tokio::io;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpStream;
use tokio::prelude::*;
pub fn main() -> Result<(), Box<std::error::Error>> {
let addr = "127.0.0.1:6142".parse()?;
use std::error::Error;
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn Error>> {
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpStream, which is fully async.
let client = TcpStream::connect(&addr).and_then(|stream| {
println!("created stream");
io::write_all(stream, "hello world\n").then(|result| {
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
})
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("connection error = {:?}", err);
});
let mut stream = TcpStream::connect("127.0.0.1:6142").await?;
println!("created stream");
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
println!("About to create the stream and write to it...");
tokio::run(client);
println!("Stream has been created and written to.");
let result = stream.write(b"hello world\n").await;
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
}
-87
View File
@@ -1,87 +0,0 @@
//! An example how to manually assemble a runtime and run some tasks on it.
//!
//! This is closer to the single-threaded runtime than the default tokio one, as it is simpler to
//! grasp. There are conceptually similar, but the multi-threaded one would be more code. If you
//! just want to *use* a single-threaded runtime, use the one provided by tokio directly
//! (`tokio::runtime::current_thread::Runtime::new()`. This is a demonstration only.
//!
//! Note that the error handling is a bit left out. Also, the `run` could be modified to return the
//! result of the provided future.
extern crate futures;
extern crate tokio;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_timer;
use std::io::Error as IoError;
use std::time::{Duration, Instant};
use futures::{future, Future};
use tokio_current_thread::CurrentThread;
use tokio_reactor::Reactor;
use tokio_timer::timer::{self, Timer};
/// Creates a "runtime".
///
/// This is similar to running `tokio::runtime::current_thread::Runtime::new()`.
fn run<F: Future<Item = (), Error = ()>>(f: F) -> Result<(), IoError> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new(reactor);
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor generate some new stimuli for the
// futures to continue in their life.
let mut executor = CurrentThread::new_with_park(timer);
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the current single-threaded
// executor when used. This is a trick, because we need two mutable references to the
// executor (one to run the provided future, another to install as the default one). We
// use the fake one here as the default one.
let mut default_executor = tokio_current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
// Run the provided future
executor.block_on(f).unwrap();
// Run all the other futures that are still left in the executor
executor.run().unwrap();
});
});
});
Ok(())
}
fn main() -> Result<(), Box<std::error::Error>> {
run(future::lazy(|| {
// Here comes the application logic. It can spawn further tasks by tokio_current_thread::spawn().
// It also can use the default reactor and create timeouts.
// Connect somewhere. And then do nothing with it. Yes, useless.
//
// This will use the default reactor which runs in the current thread.
let connect = tokio::net::TcpStream::connect(&"127.0.0.1:53".parse().unwrap())
.map(|_| println!("Connected"))
.map_err(|e| println!("Failed to connect: {}", e));
// We can spawn it without requiring Send. This would panic if we run it outside of the
// `run` (or outside of anything else)
tokio_current_thread::spawn(connect);
// We can also create timeouts.
let deadline = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(5))
.map(|()| println!("5 seconds are over"))
.map_err(|e| println!("Failed to wait: {}", e));
// We can spawn on the default executor, which is also the local one.
tokio::executor::spawn(deadline);
Ok(())
}))?;
Ok(())
}
+29 -75
View File
@@ -52,99 +52,53 @@
//! ```
//!
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
extern crate tokio_codec;
use tokio_codec::BytesCodec;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::codec::Decoder;
use tokio_util::codec::{BytesCodec, Decoder};
use futures::StreamExt;
use std::env;
use std::net::SocketAddr;
fn main() -> Result<(), Box<std::error::Error>> {
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr)?;
let mut listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We're parsing each socket with the `BytesCodec` included in `tokio_io`,
// and then we `split` each codec into the reader/writer halves.
//
// See https://docs.rs/tokio-codec/0.1/src/tokio_codec/bytes_codec.rs.html
let framed = BytesCodec::new().framed(socket);
let (_writer, reader) = framed.split();
loop {
// Asynchronously wait for an inbound socket.
let (socket, _) = listener.accept().await?;
let processor = reader
.for_each(|bytes| {
println!("bytes: {:?}", bytes);
Ok(())
})
// After our copy operation is complete we just print out some helpful
// information.
.and_then(|()| {
println!("Socket received FIN packet and closed connection");
Ok(())
})
.or_else(|err| {
println!("Socket closed with error: {:?}", err);
// We have to return the error to catch it in the next ``.then` call
Err(err)
})
.then(|result| {
println!("Socket closed with result: {:?}", result);
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(async move {
// We're parsing each socket with the `BytesCodec` included in `tokio::codec`.
let mut framed = BytesCodec::new().framed(socket);
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(processor)
// We loop while there are messages coming from the Stream `framed`.
// The stream will return None once the client disconnects.
while let Some(message) = framed.next().await {
match message {
Ok(bytes) => println!("bytes: {:?}", bytes),
Err(err) => println!("Socket closed with error: {:?}", err),
}
}
println!("Socket received FIN packet and closed connection");
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
}
+29 -90
View File
@@ -20,110 +20,49 @@
//! This final terminal will connect to our proxy, which will in turn connect to
//! the echo server, and you'll be able to see data flowing between them.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use tokio::io::{copy, shutdown};
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
fn main() -> Result<(), Box<std::error::Error>> {
use futures::future::try_join;
use futures::FutureExt;
use std::env;
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let listen_addr = env::args().nth(1).unwrap_or("127.0.0.1:8081".to_string());
let listen_addr = listen_addr.parse::<SocketAddr>()?;
let server_addr = env::args().nth(2).unwrap_or("127.0.0.1:8080".to_string());
let server_addr = server_addr.parse::<SocketAddr>()?;
// Create a TCP listener which will listen for incoming connections.
let socket = TcpListener::bind(&listen_addr)?;
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
let amounts = server.and_then(move |server| {
// Create separate read/write handles for the TCP clients that we're
// proxying data between. Note that typically you'd use
// `AsyncRead::split` for this operation, but we want our writer
// handles to have a custom implementation of `shutdown` which
// actually calls `TcpStream::shutdown` to ensure that EOF is
// transmitted properly across the proxied connection.
//
// As a result, we wrap up our client/server manually in arcs and
// use the impls below on our custom `MyTcpStream` type.
let client_reader = MyTcpStream(Arc::new(Mutex::new(client)));
let client_writer = client_reader.clone();
let server_reader = MyTcpStream(Arc::new(Mutex::new(server)));
let server_writer = server_reader.clone();
let mut listener = TcpListener::bind(listen_addr).await?;
// Copy the data (in parallel) between the client and the server.
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
Ok(())
while let Ok((inbound, _)) = listener.accept().await {
let transfer = transfer(inbound, server_addr.clone()).map(|r| {
if let Err(e) = r {
println!("Failed to transfer; error={}", e);
}
});
tokio::run(done);
tokio::spawn(transfer);
}
Ok(())
}
// This is a custom type used to have a custom implementation of the
// `AsyncWrite::shutdown` method which actually calls `TcpStream::shutdown` to
// notify the remote end that we're done writing.
#[derive(Clone)]
struct MyTcpStream(Arc<Mutex<TcpStream>>);
async fn transfer(mut inbound: TcpStream, proxy_addr: String) -> Result<(), Box<dyn Error>> {
let mut outbound = TcpStream::connect(proxy_addr).await?;
impl Read for MyTcpStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.0.lock().unwrap().read(buf)
}
}
impl Write for MyTcpStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().unwrap().write(buf)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl AsyncRead for MyTcpStream {}
impl AsyncWrite for MyTcpStream {
fn shutdown(&mut self) -> Poll<(), io::Error> {
try!(self.0.lock().unwrap().shutdown(Shutdown::Write));
Ok(().into())
}
let (mut ri, mut wi) = inbound.split();
let (mut ro, mut wo) = outbound.split();
let client_to_server = io::copy(&mut ri, &mut wo);
let server_to_client = io::copy(&mut ro, &mut wi);
try_join(client_to_server, server_to_client).await?;
Ok(())
}
+102 -85
View File
@@ -39,19 +39,16 @@
//! * `SET $key $value` - this will set the value of `$key` to `$value`,
//! returning the previous value, if any.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use tokio::io::{lines, write_all};
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio_util::codec::{Framed, LinesCodec};
use futures::{SinkExt, StreamExt};
use std::collections::HashMap;
use std::env;
use std::error::Error;
use std::sync::{Arc, Mutex};
/// The in-memory database shared amongst all clients.
///
@@ -69,17 +66,27 @@ enum Request {
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
Value {
key: String,
value: String,
},
Set {
key: String,
value: String,
previous: Option<String>,
},
Error {
msg: String,
},
}
fn main() -> Result<(), Box<std::error::Error>> {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr).map_err(|_| "failed to bind")?;
let mut listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
@@ -93,70 +100,77 @@ fn main() -> Result<(), Box<std::error::Error>> {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
// *split* this TCP stream into two separately owned halves. This'll
// allow us to work with the read and write halves independently.
let (reader, writer) = socket.split();
loop {
match listener.accept().await {
Ok((socket, _)) => {
// After getting a new connection first we see a clone of the database
// being created, which is creating a new reference for this connected
// client to use.
let db = db.clone();
// Since our protocol is line-based we use `tokio_io`'s `lines` utility
// to convert our stream of bytes, `reader`, into a `Stream` of lines.
let lines = lines(BufReader::new(reader));
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients. The `move` keyword is used
// here to move ownership of our db handle into the async closure.
tokio::spawn(async move {
// Since our protocol is line-based we use `tokio_codecs`'s `LineCodec`
// to convert our stream of bytes, `socket`, into a `Stream` of lines
// as well as convert our line based responses into a stream of bytes.
let mut lines = Framed::new(socket, LinesCodec::new());
// Here's where the meat of the processing in this server happens. First
// we see a clone of the database being created, which is creating a
// new reference for this connected client to use. Also note the `move`
// keyword on the closure here which moves ownership of the reference
// into the closure, which we'll need for spawning the client below.
//
// The `map` function here means that we'll run some code for all
// requests (lines) we receive from the client. The actual handling here
// is pretty simple, first we parse the request and if it's valid we
// generate a response based on the values in the database.
let db = db.clone();
let responses = lines.map(move |line| {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
// Here for every line we get back from the `Framed` decoder,
// we parse the request, and if it's valid we generate a response
// based on the values in the database.
while let Some(result) = lines.next().await {
match result {
Ok(line) => {
let response = handle_request(&line, &db);
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
let response = response.serialize();
if let Err(e) = lines.send(response).await {
println!("error on sending response; error = {:?}", e);
}
}
Err(e) => {
println!("error on decoding from socket; error = {:?}", e);
}
}
}
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
}
}
});
// At this point `responses` is a stream of `Response` types which we
// now want to write back out to the client. To do that we use
// `Stream::fold` to perform a loop here, serializing each response and
// then writing it out to the client.
let writes = responses.fold(writer, |writer, response| {
let mut response = response.serialize();
response.push('\n');
write_all(writer, response.into_bytes()).map(|(w, _)| w)
});
// The connection will be closed at this point as `lines.next()` has returned `None`.
});
}
Err(e) => println!("error accepting socket; error = {:?}", e),
}
}
}
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients, for now ignoring any errors
// that we see.
let msg = writes.then(move |_| Ok(()));
fn handle_request(line: &str, db: &Arc<Database>) -> Response {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
tokio::spawn(msg)
});
tokio::run(done);
Ok(())
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => match db.get(&key) {
Some(value) => Response::Value {
key,
value: value.clone(),
},
None => Response::Error {
msg: format!("no key {}", key),
},
},
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set {
key,
value,
previous,
}
}
}
}
impl Request {
@@ -169,9 +183,11 @@ impl Request {
None => return Err(format!("GET must be followed by a key")),
};
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
return Err(format!("GET's key must not be followed by anything"));
}
Ok(Request::Get { key: key.to_string() })
Ok(Request::Get {
key: key.to_string(),
})
}
Some("SET") => {
let key = match parts.next() {
@@ -182,7 +198,10 @@ impl Request {
Some(value) => value,
None => return Err(format!("SET needs a value")),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
Ok(Request::Set {
key: key.to_string(),
value: value.to_string(),
})
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
@@ -193,15 +212,13 @@ impl Request {
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
Response::Value { ref key, ref value } => format!("{} = {}", key, value),
Response::Set {
ref key,
ref value,
ref previous,
} => format!("set {} = `{}`, previous: {:?}", key, value, previous),
Response::Error { ref msg } => format!("error: {}", msg),
}
}
}
+87 -101
View File
@@ -11,106 +11,82 @@
//! respectively. By default this will run I/O on all the cores your system has
//! available, and it doesn't support HTTP request bodies.
#![deny(warnings)]
extern crate bytes;
extern crate http;
extern crate httparse;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
extern crate time;
extern crate tokio;
extern crate tokio_io;
use std::{env, fmt, io};
use std::net::SocketAddr;
use tokio::net::{TcpStream, TcpListener};
use tokio::prelude::*;
use tokio::codec::{Encoder, Decoder};
#![warn(rust_2018_idioms)]
use bytes::BytesMut;
use http::header::HeaderValue;
use http::{Request, Response, StatusCode};
use futures::{SinkExt, StreamExt};
use http::{header::HeaderValue, Request, Response, StatusCode};
use serde::Serialize;
use std::{env, error::Error, fmt, io};
use tokio::{
codec::{Decoder, Encoder, Framed},
net::{TcpListener, TcpStream},
};
fn main() -> Result<(), Box<std::error::Error>> {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Parse the arguments, bind the TCP socket we'll be listening to, spin up
// our worker threads, and start shipping sockets to those worker threads.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr)?;
let mut incoming = TcpListener::bind(&addr).await?.incoming();
println!("Listening on: {}", addr);
tokio::run({
listener.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(|socket| {
process(socket);
Ok(())
})
});
while let Some(Ok(stream)) = incoming.next().await {
tokio::spawn(async move {
if let Err(e) = process(stream).await {
println!("failed to process connection; error = {}", e);
}
});
}
Ok(())
}
fn process(socket: TcpStream) {
let (tx, rx) =
// Frame the socket using the `Http` protocol. This maps the TCP socket
// to a Stream + Sink of HTTP frames.
Http.framed(socket)
// This splits a single `Stream + Sink` value into two separate handles
// that can be used independently (even on different tasks or threads).
.split();
async fn process(stream: TcpStream) -> Result<(), Box<dyn Error>> {
let mut transport = Framed::new(stream, Http);
// Map all requests into responses and send them back to the client.
let task = tx.send_all(rx.and_then(respond))
.then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
while let Some(request) = transport.next().await {
match request {
Ok(request) => {
let response = respond(request).await?;
transport.send(response).await?;
}
Err(e) => return Err(e.into()),
}
}
Ok(())
});
// Spawn the task that handles the connection.
tokio::spawn(task);
Ok(())
}
/// "Server logic" is implemented in this function.
///
/// This function is a map from and HTTP request to a future of a response and
/// represents the various handling a server might do. Currently the contents
/// here are pretty uninteresting.
fn respond(req: Request<()>)
-> Box<Future<Item = Response<String>, Error = io::Error> + Send>
{
let f = future::lazy(move || {
let mut response = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
response.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
"/json" => {
response.header("Content-Type", "application/json");
async fn respond(req: Request<()>) -> Result<Response<String>, Box<dyn Error>> {
let mut response = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
response.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
"/json" => {
response.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
}
serde_json::to_string(&Message { message: "Hello, World!" })?
#[derive(Serialize)]
struct Message {
message: &'static str,
}
_ => {
response.status(StatusCode::NOT_FOUND);
String::new()
}
};
let response = response.body(body).map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Ok(response)
});
serde_json::to_string(&Message {
message: "Hello, World!",
})?
}
_ => {
response.status(StatusCode::NOT_FOUND);
String::new()
}
};
let response = response
.body(body)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Box::new(f)
Ok(response)
}
struct Http;
@@ -124,12 +100,19 @@ impl Encoder for Http {
fn encode(&mut self, item: Response<String>, dst: &mut BytesMut) -> io::Result<()> {
use std::fmt::Write;
write!(BytesWrite(dst), "\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
", item.status(), item.body().len(), date::now()).unwrap();
write!(
BytesWrite(dst),
"\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
",
item.status(),
item.body().len(),
date::now()
)
.unwrap();
for (k, v) in item.headers() {
dst.extend_from_slice(k.as_str().as_bytes());
@@ -148,13 +131,13 @@ impl Encoder for Http {
// doesn't go through io::Error.
struct BytesWrite<'a>(&'a mut BytesMut);
impl<'a> fmt::Write for BytesWrite<'a> {
impl fmt::Write for BytesWrite<'_> {
fn write_str(&mut self, s: &str) -> fmt::Result {
self.0.extend_from_slice(s.as_bytes());
Ok(())
}
fn write_fmt(&mut self, args: fmt::Arguments) -> fmt::Result {
fn write_fmt(&mut self, args: fmt::Arguments<'_>) -> fmt::Result {
fmt::write(self, args)
}
}
@@ -198,13 +181,18 @@ impl Decoder for Http {
headers[i] = Some((k, v));
}
(toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt)
(
toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt,
)
};
if version != 1 {
return Err(io::Error::new(io::ErrorKind::Other, "only HTTP/1.1 accepted"))
return Err(io::Error::new(
io::ErrorKind::Other,
"only HTTP/1.1 accepted",
));
}
let data = src.split_to(amt).freeze();
let mut ret = Request::builder();
@@ -216,15 +204,13 @@ impl Decoder for Http {
Some((ref k, ref v)) => (k, v),
None => break,
};
let value = unsafe {
HeaderValue::from_shared_unchecked(data.slice(v.0, v.1))
};
let value = unsafe { HeaderValue::from_shared_unchecked(data.slice(v.0, v.1)) };
ret.header(&data[k.0..k.1], value);
}
let req = ret.body(()).map_err(|e| {
io::Error::new(io::ErrorKind::Other, e)
})?;
let req = ret
.body(())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
Ok(Some(req))
}
}
@@ -274,7 +260,7 @@ mod date {
}));
impl fmt::Display for Now {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
LAST.with(|cache| {
let mut cache = cache.borrow_mut();
let now = time::get_time();
@@ -301,7 +287,7 @@ mod date {
struct LocalBuffer<'a>(&'a mut LastRenderedNow);
impl<'a> fmt::Write for LocalBuffer<'a> {
impl fmt::Write for LocalBuffer<'_> {
fn write_str(&mut self, s: &str) -> fmt::Result {
let start = self.0.amt;
let end = start + s.len();
+22 -19
View File
@@ -26,44 +26,47 @@
//! Please mind that since the UDP protocol doesn't have any capabilities to detect a broken
//! connection the server needs to be run first, otherwise the client will block forever.
extern crate futures;
extern crate tokio;
#![warn(rust_2018_idioms)]
use std::env;
use std::io::stdin;
use std::error::Error;
use std::io::{stdin, Read};
use std::net::SocketAddr;
use tokio::net::UdpSocket;
use tokio::prelude::*;
fn get_stdin_data() -> Result<Vec<u8>, Box<std::error::Error>> {
fn get_stdin_data() -> Result<Vec<u8>, Box<dyn std::error::Error>> {
let mut buf = Vec::new();
stdin().read_to_end(&mut buf)?;
Ok(buf)
}
fn main() -> Result<(), Box<std::error::Error>> {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let remote_addr: SocketAddr = env::args()
.nth(1)
.unwrap_or("127.0.0.1:8080".into())
.parse()?;
// We use port 0 to let the operating system allocate an available port for us.
let local_addr: SocketAddr = if remote_addr.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
}.parse()?;
let socket = UdpSocket::bind(&local_addr)?;
}
.parse()?;
let mut socket = UdpSocket::bind(local_addr).await?;
const MAX_DATAGRAM_SIZE: usize = 65_507;
socket
.send_dgram(get_stdin_data()?, &remote_addr)
.and_then(|(socket, _)| socket.recv_dgram(vec![0u8; MAX_DATAGRAM_SIZE]))
.map(|(_, data, len, _)| {
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
)
})
.wait()?;
socket.connect(&remote_addr).await?;
let data = get_stdin_data()?;
socket.send(&data).await?;
let mut data = vec![0u8; MAX_DATAGRAM_SIZE];
let len = socket.recv(&mut data).await?;
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
);
Ok(())
}
+53 -41
View File
@@ -1,65 +1,77 @@
//! This example leverages `BytesCodec` to create a UDP client and server which
//! speak a custom protocol.
//!
//! Here we're using the codec from tokio-io to convert a UDP socket to a stream of
//! Here we're using the codec from `tokio-codec` to convert a UDP socket to a stream of
//! client messages. These messages are then processed and returned back as a
//! new message with a new destination. Overall, we then use this to construct a
//! "ping pong" pair where two sockets are sending messages back and forth.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate env_logger;
use tokio::net::UdpSocket;
use tokio::{io, time};
use tokio_util::codec::BytesCodec;
use tokio_util::udp::UdpFramed;
use bytes::Bytes;
use futures::{FutureExt, SinkExt, StreamExt};
use std::env;
use std::error::Error;
use std::net::SocketAddr;
use std::time::Duration;
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_codec::BytesCodec;
fn main() -> Result<(), Box<std::error::Error>> {
let _ = env_logger::init();
let addr: SocketAddr = "127.0.0.1:0".parse()?;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let addr = env::args().nth(1).unwrap_or("127.0.0.1:0".to_string());
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr)?;
let b = UdpSocket::bind(&addr)?;
let a = UdpSocket::bind(&addr).await?;
let b = UdpSocket::bind(&addr).await?;
let b_addr = b.local_addr()?;
// We're parsing each socket with the `BytesCodec` included in `tokio_io`, and then we
// `split` each codec into the sink/stream halves.
let (a_sink, a_stream) = UdpFramed::new(a, BytesCodec::new()).split();
let (b_sink, b_stream) = UdpFramed::new(b, BytesCodec::new()).split();
let mut a = UdpFramed::new(a, BytesCodec::new());
let mut b = UdpFramed::new(b, BytesCodec::new());
// Start off by sending a ping from a to b, afterwards we just print out
// what they send us and continually send pings
// let pings = stream::iter((0..5).map(Ok));
let a = a_sink.send(("PING".into(), b_addr)).and_then(|a_sink| {
let mut i = 0;
let a_stream = a_stream.take(4).map(move |(msg, addr)| {
i += 1;
println!("[a] recv: {}", String::from_utf8_lossy(&msg));
(format!("PING {}", i).into(), addr)
});
a_sink.send_all(a_stream)
});
let a = ping(&mut a, b_addr);
// The second client we have will receive the pings from `a` and then send
// back pongs.
let b_stream = b_stream.map(|(msg, addr)| {
println!("[b] recv: {}", String::from_utf8_lossy(&msg));
("PONG".into(), addr)
});
let b = b_sink.send_all(b_stream);
let b = pong(&mut b);
// Run both futures simultaneously of `a` and `b` sending messages back and forth.
match futures::future::try_join(a, b).await {
Err(e) => println!("an error occured; error = {:?}", e),
_ => println!("done!"),
}
Ok(())
}
async fn ping(socket: &mut UdpFramed<BytesCodec>, b_addr: SocketAddr) -> Result<(), io::Error> {
socket.send((Bytes::from(&b"PING"[..]), b_addr)).await?;
for _ in 0..4usize {
let (bytes, addr) = socket.next().map(|e| e.unwrap()).await?;
println!("[a] recv: {}", String::from_utf8_lossy(&bytes));
socket.send((Bytes::from(&b"PING"[..]), addr)).await?;
}
Ok(())
}
async fn pong(socket: &mut UdpFramed<BytesCodec>) -> Result<(), io::Error> {
let timeout = Duration::from_millis(200);
while let Ok(Some(Ok((bytes, addr)))) = time::timeout(timeout, socket.next()).await {
println!("[b] recv: {}", String::from_utf8_lossy(&bytes));
socket.send((Bytes::from(&b"PONG"[..]), addr)).await?;
}
// Spawn the sender of pongs and then wait for our pinger to finish.
tokio::run({
b.join(a)
.map(|_| ())
.map_err(|e| println!("error = {:?}", e))
});
Ok(())
}
+1
View File
@@ -0,0 +1 @@
edition = "2018"
-26
View File
@@ -1,26 +0,0 @@
use std::future::{Future as StdFuture};
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
-15
View File
@@ -1,15 +0,0 @@
//! A configurable source of time.
//!
//! This module provides the [`now`][n] function, which returns an `Instant`
//! representing "now". The source of time used by this function is configurable
//! (via the [`tokio-timer`] crate) and allows mocking out the source of time in
//! tests or performing caching operations to reduce the number of syscalls.
//!
//! Note that, because the source of time is configurable, it is possible to
//! observe non-monotonic behavior when calling [`now`][n] from different
//! executors.
//!
//! [n]: fn.now.html
//! [`tokio-timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/clock/index.html
pub use tokio_timer::clock::now;
-26
View File
@@ -1,26 +0,0 @@
//! Utilities for encoding and decoding frames.
//!
//! Contains adapters to go from streams of bytes, [`AsyncRead`] and
//! [`AsyncWrite`], to framed streams implementing [`Sink`] and [`Stream`].
//! Framed streams are also known as [transports].
//!
//! [`AsyncRead`]: ../io/trait.AsyncRead.html
//! [`AsyncWrite`]: ../io/trait.AsyncWrite.html
//! [`Sink`]: https://docs.rs/futures/0.1/futures/sink/trait.Sink.html
//! [`Stream`]: https://docs.rs/futures/0.1/futures/stream/trait.Stream.html
//! [transports]: https://tokio.rs/docs/going-deeper/frames/
pub use tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
BytesCodec,
LinesCodec,
};
pub mod length_delimited;
pub use self::length_delimited::LengthDelimitedCodec;
-170
View File
@@ -1,170 +0,0 @@
#![allow(deprecated)]
//! Execute many tasks concurrently on the current thread.
//!
//! [`CurrentThread`] is an executor that keeps tasks on the same thread that
//! they were spawned from. This allows it to execute futures that are not
//! `Send`.
//!
//! A single [`CurrentThread`] instance is able to efficiently manage a large
//! number of tasks and will attempt to schedule all tasks fairly.
//!
//! All tasks that are being managed by a [`CurrentThread`] executor are able to
//! spawn additional tasks by calling [`spawn`]. This function only works from
//! within the context of a running [`CurrentThread`] instance.
//!
//! The easiest way to start a new [`CurrentThread`] executor is to call
//! [`block_on_all`] with an initial task to seed the executor.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread;
//! use futures::future::lazy;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! current_thread::block_on_all(lazy(|| {
//! // The execution context is setup, futures may be executed.
//! current_thread::spawn(lazy(|| {
//! println!("called from the current thread executor");
//! Ok(())
//! }));
//!
//! Ok::<_, ()>(())
//! }));
//! # }
//! ```
//!
//! The `block_on_all` function will block the current thread until **all**
//! tasks that have been spawned onto the [`CurrentThread`] instance have
//! completed.
//!
//! More fine-grain control can be achieved by using [`CurrentThread`] directly.
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread::CurrentThread;
//! use futures::future::{lazy, empty};
//! use std::time::Duration;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! let mut current_thread = CurrentThread::new();
//!
//! // Spawn a task, the task is not executed yet.
//! current_thread.spawn(lazy(|| {
//! println!("Spawning a task");
//! Ok(())
//! }));
//!
//! // Spawn a task that never completes
//! current_thread.spawn(empty());
//!
//! // Run the executor, but only until the provided future completes. This
//! // provides the opportunity to start executing previously spawned tasks.
//! let res = current_thread.block_on(lazy(|| {
//! Ok::<_, ()>("Hello")
//! })).unwrap();
//!
//! // Now, run the executor for *at most* 1 second. Since a task was spawned
//! // that never completes, this function will return with an error.
//! current_thread.run_timeout(Duration::from_secs(1)).unwrap_err();
//! # }
//! ```
//!
//! # Execution model
//!
//! Internally, [`CurrentThread`] maintains a queue. When one of its tasks is
//! notified, the task gets added to the queue. The executor will pop tasks from
//! the queue and call [`Future::poll`]. If the task gets notified while it is
//! being executed, it won't get re-executed until all other tasks currently in
//! the queue get polled.
//!
//! Before the task is polled, a thread-local variable referencing the current
//! [`CurrentThread`] instance is set. This enables [`spawn`] to spawn new tasks
//! onto the same executor without having to thread through a handle value.
//!
//! If the [`CurrentThread`] instance still has uncompleted tasks, but none of
//! these tasks are ready to be polled, the current thread is put to sleep. When
//! a task is notified, the thread is woken up and processing resumes.
//!
//! All tasks managed by [`CurrentThread`] remain on the current thread. When a
//! task completes, it is dropped.
//!
//! [`spawn`]: fn.spawn.html
//! [`block_on_all`]: fn.block_on_all.html
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
pub use tokio_current_thread::{
BlockError,
CurrentThread,
Entered,
Handle,
RunError,
RunTimeoutError,
TaskExecutor,
Turn,
TurnError,
block_on_all,
spawn,
};
use std::cell::Cell;
use std::marker::PhantomData;
use futures::future::{self};
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[derive(Debug)]
pub struct Context<'a> {
cancel: Cell<bool>,
_p: PhantomData<&'a ()>,
}
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
pub fn run<F, R>(f: F) -> R
where F: FnOnce(&mut Context) -> R
{
let mut context = Context {
cancel: Cell::new(false),
_p: PhantomData,
};
let mut current_thread = CurrentThread::new();
let ret = current_thread
.block_on(future::lazy(|| Ok::<_, ()>(f(&mut context))))
.unwrap();
if context.cancel.get() {
return ret;
}
current_thread.run().unwrap();
ret
}
#[deprecated(since = "0.1.2", note = "use TaskExecutor::current instead")]
#[doc(hidden)]
pub fn task_executor() -> TaskExecutor {
TaskExecutor::current()
}
-145
View File
@@ -1,145 +0,0 @@
//! Task execution utilities.
//!
//! In the Tokio execution model, futures are lazy. When a future is created, no
//! work is performed. In order for the work defined by the future to happen,
//! the future must be submitted to an executor. A future that is submitted to
//! an executor is called a "task".
//!
//! The executor is responsible for ensuring that [`Future::poll`] is
//! called whenever the task is [notified]. Notification happens when the
//! internal state of a task transitions from "not ready" to ready. For
//! example, a socket might have received data and a call to `read` will now be
//! able to succeed.
//!
//! The specific strategy used to manage the tasks is left up to the
//! executor. There are two main flavors of executors: single-threaded and
//! multi-threaded. Tokio provides implementation for both of these in the
//! [`runtime`] module.
//!
//! # `Executor` trait.
//!
//! This module provides the [`Executor`] trait (re-exported from
//! [`tokio-executor`]), which describes the API that all executors must
//! implement.
//!
//! A free [`spawn`] function is provided that allows spawning futures onto the
//! default executor (tracked via a thread-local variable) without referencing a
//! handle. It is expected that all executors will set a value for the default
//! executor. This value will often be set to the executor itself, but it is
//! possible that the default executor might be set to a different executor.
//!
//! For example, a single threaded executor might set the default executor to a
//! thread pool instead of itself, allowing futures to spawn new tasks onto the
//! thread pool when those tasks are `Send`.
//!
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
//! [notified]: https://docs.rs/futures/0.1/futures/executor/trait.Notify.html#tymethod.notify
//! [`runtime`]: ../runtime/index.html
//! [`tokio-executor`]: https://docs.rs/tokio-executor/0.1
//! [`Executor`]: trait.Executor.html
//! [`spawn`]: fn.spawn.html
#[deprecated(
since = "0.1.8",
note = "use tokio-current-thread crate or functions in tokio::runtime::current_thread instead",
)]
#[doc(hidden)]
pub mod current_thread;
#[deprecated(since = "0.1.8", note = "use tokio-threadpool crate instead")]
#[doc(hidden)]
/// Re-exports of [`tokio-threadpool`], deprecated in favor of the crate.
///
/// [`tokio-threadpool`]: https://docs.rs/tokio-threadpool/0.1
pub mod thread_pool {
pub use tokio_threadpool::{
Builder,
Sender,
Shutdown,
ThreadPool,
};
}
pub use tokio_executor::{Executor, DefaultExecutor, SpawnError};
use futures::{Future, IntoFuture};
use futures::future::{self, FutureResult};
/// Return value from the `spawn` function.
///
/// Currently this value doesn't actually provide any functionality. However, it
/// provides a way to add functionality later without breaking backwards
/// compatibility.
///
/// This also implements `IntoFuture` so that it can be used as the return value
/// in a `for_each` loop.
///
/// See [`spawn`] for more details.
///
/// [`spawn`]: fn.spawn.html
#[derive(Debug)]
pub struct Spawn(());
/// Spawns a future on the default executor.
///
/// In order for a future to do work, it must be spawned on an executor. The
/// `spawn` function is the easiest way to do this. It spawns a future on the
/// [default executor] for the current execution context (tracked using a
/// thread-local variable).
///
/// The default executor is **usually** a thread pool.
///
/// # Examples
///
/// In this example, a server is started and `spawn` is used to start a new task
/// that processes each received connection.
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// [default executor]: struct.DefaultExecutor.html
///
/// # Panics
///
/// This function will panic if the default executor is not set or if spawning
/// onto the default executor returns an error. To avoid the panic, use
/// [`DefaultExecutor`].
///
/// [`DefaultExecutor`]: struct.DefaultExecutor.html
pub fn spawn<F>(f: F) -> Spawn
where F: Future<Item = (), Error = ()> + 'static + Send
{
::tokio_executor::spawn(f);
Spawn(())
}
impl IntoFuture for Spawn {
type Future = FutureResult<(), ()>;
type Item = ();
type Error = ();
fn into_future(self) -> Self::Future {
future::ok(())
}
}
-12
View File
@@ -1,12 +0,0 @@
//! Asynchronous filesystem manipulation operations.
//!
//! This module contains basic methods and types for manipulating the contents
//! of the local filesystem from within the context of the Tokio runtime.
//!
//! Unlike *most* other Tokio APIs, the filesystem APIs **must** be used from
//! the context of the Tokio runtime as they require Tokio specific features to
//! function.
pub use tokio_fs::{create_dir, create_dir_all, file, hard_link, metadata, os, read_dir, read_link};
pub use tokio_fs::{remove_dir, remove_file, rename, set_permissions, symlink_metadata, File};
pub use tokio_fs::OpenOptions;
-95
View File
@@ -1,95 +0,0 @@
//! Asynchronous I/O.
//!
//! This module is the asynchronous version of `std::io`. Primarily, it
//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which extend the
//! `Read` and `Write` traits of the standard library.
//!
//! # AsyncRead and AsyncWrite
//!
//! [`AsyncRead`] and [`AsyncWrite`] must only be implemented for
//! non-blocking I/O types that integrate with the futures type system. In
//! other words, these types must never block the thread, and instead the
//! current task is notified when the I/O resource is ready.
//!
//! # Standard input and output
//!
//! Tokio provides asynchronous APIs to standard [input], [output], and [error].
//! These APIs are very similar to the ones provided by `std`, but they also
//! implement [`AsyncRead`] and [`AsyncWrite`].
//!
//! Unlike *most* other Tokio APIs, the standard input / output APIs
//! **must** be used from the context of the Tokio runtime as they require
//! Tokio specific features to function.
//!
//! [input]: fn.stdin.html
//! [output]: fn.stdout.html
//! [error]: fn.stderr.html
//!
//! # Utility functions
//!
//! Utilities functions are provided for working with [`AsyncRead`] /
//! [`AsyncWrite`] types. For example, [`copy`] asynchronously copies all
//! data from a source to a destination.
//!
//! # `std` re-exports
//!
//! Additionally, [`Read`], [`Write`], [`Error`], [`ErrorKind`], and
//! [`Result`] are re-exported from `std::io` for ease of use.
//!
//! [`AsyncRead`]: trait.AsyncRead.html
//! [`AsyncWrite`]: trait.AsyncWrite.html
//! [`copy`]: fn.copy.html
//! [`Read`]: trait.Read.html
//! [`Write`]: trait.Write.html
//! [`Error`]: struct.Error.html
//! [`ErrorKind`]: enum.ErrorKind.html
//! [`Result`]: type.Result.html
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
// standard input, output, and error
#[cfg(feature = "fs")]
pub use tokio_fs::{
stdin,
Stdin,
stdout,
Stdout,
stderr,
Stderr,
};
// Utils
pub use tokio_io::io::{
copy,
Copy,
flush,
Flush,
lines,
Lines,
read,
read_exact,
ReadExact,
read_to_end,
ReadToEnd,
read_until,
ReadUntil,
ReadHalf,
shutdown,
Shutdown,
write_all,
WriteAll,
WriteHalf,
};
// Re-export io::Error so that users don't have to deal
// with conflicts when `use`ing `futures::io` and `std::io`.
pub use ::std::io::{
Error,
ErrorKind,
Result,
Read,
Write,
};
-155
View File
@@ -1,155 +0,0 @@
#![doc(html_root_url = "https://docs.rs/tokio/0.1.15")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
#![cfg_attr(feature = "async-await-preview", feature(
async_await,
await_macro,
futures_api,
))]
//! A runtime for writing reliable, asynchronous, and slim applications.
//!
//! Tokio is an event-driven, non-blocking I/O platform for writing asynchronous
//! applications with the Rust programming language. At a high level, it
//! provides a few major components:
//!
//! * A multi threaded, work-stealing based task [scheduler][runtime].
//! * A [reactor] backed by the operating system's event queue (epoll, kqueue,
//! IOCP, etc...).
//! * Asynchronous [TCP and UDP][net] sockets.
//! * Asynchronous [filesystem][fs] operations.
//! * [Timer][timer] API for scheduling work in the future.
//!
//! Tokio is built using [futures] as the abstraction for managing the
//! complexity of asynchronous programming.
//!
//! Guide level documentation is found on the [website].
//!
//! [website]: https://tokio.rs/docs/getting-started/hello-world/
//! [futures]: http://docs.rs/futures/0.1
//!
//! # Examples
//!
//! A simple TCP echo server:
//!
//! ```no_run
//! extern crate tokio;
//!
//! use tokio::prelude::*;
//! use tokio::io::copy;
//! use tokio::net::TcpListener;
//!
//! fn main() {
//! // Bind the server's socket.
//! let addr = "127.0.0.1:12345".parse().unwrap();
//! let listener = TcpListener::bind(&addr)
//! .expect("unable to bind TCP listener");
//!
//! // Pull out a stream of sockets for incoming connections
//! let server = listener.incoming()
//! .map_err(|e| eprintln!("accept failed = {:?}", e))
//! .for_each(|sock| {
//! // Split up the reading and writing parts of the
//! // socket.
//! let (reader, writer) = sock.split();
//!
//! // A future that echos the data and returns how
//! // many bytes were copied...
//! let bytes_copied = copy(reader, writer);
//!
//! // ... after which we'll print what happened.
//! let handle_conn = bytes_copied.map(|amt| {
//! println!("wrote {:?} bytes", amt)
//! }).map_err(|err| {
//! eprintln!("IO error {:?}", err)
//! });
//!
//! // Spawn the future as a concurrent task.
//! tokio::spawn(handle_conn)
//! });
//!
//! // Start the Tokio runtime
//! tokio::run(server);
//! }
//! ```
macro_rules! if_runtime {
($($i:item)*) => ($(
#[cfg(any(feature = "rt-full"))]
$i
)*)
}
#[macro_use]
extern crate futures;
#[cfg(feature = "io")]
extern crate bytes;
#[cfg(feature = "reactor")]
extern crate mio;
#[cfg(feature = "rt-full")]
extern crate num_cpus;
#[cfg(feature = "rt-full")]
extern crate tokio_current_thread;
#[cfg(feature = "io")]
extern crate tokio_io;
#[cfg(feature = "codec")]
extern crate tokio_codec;
#[cfg(feature = "fs")]
extern crate tokio_fs;
#[cfg(feature = "reactor")]
extern crate tokio_reactor;
#[cfg(feature = "rt-full")]
extern crate tokio_threadpool;
#[cfg(feature = "sync")]
extern crate tokio_sync;
#[cfg(feature = "timer")]
extern crate tokio_timer;
#[cfg(feature = "tcp")]
extern crate tokio_tcp;
#[cfg(feature = "udp")]
extern crate tokio_udp;
#[cfg(feature = "async-await-preview")]
extern crate tokio_async_await;
#[cfg(all(unix, feature = "uds"))]
extern crate tokio_uds;
#[cfg(feature = "timer")]
pub mod clock;
#[cfg(feature = "codec")]
pub mod codec;
#[cfg(feature = "fs")]
pub mod fs;
#[cfg(feature = "io")]
pub mod io;
#[cfg(any(feature = "tcp", feature = "udp", feature = "uds"))]
pub mod net;
pub mod prelude;
#[cfg(feature = "reactor")]
pub mod reactor;
#[cfg(feature = "sync")]
pub mod sync;
#[cfg(feature = "timer")]
pub mod timer;
pub mod util;
if_runtime! {
extern crate tokio_executor;
pub mod executor;
pub mod runtime;
pub use executor::spawn;
pub use runtime::run;
}
// ===== Experimental async/await support =====
#[cfg(feature = "async-await-preview")]
mod async_await;
#[cfg(feature = "async-await-preview")]
pub use async_await::{run_async, spawn_async};
#[cfg(feature = "async-await-preview")]
pub use tokio_async_await::await;
-98
View File
@@ -1,98 +0,0 @@
//! TCP/UDP/Unix bindings for `tokio`.
//!
//! This module contains the TCP/UDP/Unix networking types, similar to the standard
//! library, which can be used to implement networking protocols.
//!
//! # Organization
//!
//! * [`TcpListener`] and [`TcpStream`] provide functionality for communication over TCP
//! * [`UdpSocket`] and [`UdpFramed`] provide functionality for communication over UDP
//! * [`UnixListener`] and [`UnixStream`] provide functionality for communication over a
//! Unix Domain Stream Socket **(available on Unix only)**
//! * [`UnixDatagram`] and [`UnixDatagramFramed`] provide functionality for communication
//! over Unix Domain Datagram Socket **(available on Unix only)**
//!
//! [`TcpListener`]: struct.TcpListener.html
//! [`TcpStream`]: struct.TcpStream.html
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`UnixListener`]: struct.UnixListener.html
//! [`UnixStream`]: struct.UnixStream.html
//! [`UnixDatagram`]: struct.UnixDatagram.html
//! [`UnixDatagramFramed`]: struct.UnixDatagramFramed.html
#[cfg(feature = "tcp")]
pub mod tcp {
//! TCP bindings for `tokio`.
//!
//! Connecting to an address, via TCP, can be done using [`TcpStream`]'s
//! [`connect`] method, which returns [`ConnectFuture`]. `ConnectFuture`
//! implements a future which returns a `TcpStream`.
//!
//! To listen on an address [`TcpListener`] can be used. `TcpListener`'s
//! [`incoming`][incoming_method] method can be used to accept new connections.
//! It return the [`Incoming`] struct, which implements a stream which returns
//! `TcpStream`s.
//!
//! [`TcpStream`]: struct.TcpStream.html
//! [`connect`]: struct.TcpStream.html#method.connect
//! [`ConnectFuture`]: struct.ConnectFuture.html
//! [`TcpListener`]: struct.TcpListener.html
//! [incoming_method]: struct.TcpListener.html#method.incoming
//! [`Incoming`]: struct.Incoming.html
pub use tokio_tcp::{ConnectFuture, Incoming, TcpListener, TcpStream};
}
#[cfg(feature = "tcp")]
pub use self::tcp::{TcpListener, TcpStream};
#[cfg(feature = "tcp")]
#[deprecated(note = "use `tokio::net::tcp::ConnectFuture` instead")]
#[doc(hidden)]
pub type ConnectFuture = self::tcp::ConnectFuture;
#[cfg(feature = "tcp")]
#[deprecated(note = "use `tokio::net::tcp::Incoming` instead")]
#[doc(hidden)]
pub type Incoming = self::tcp::Incoming;
#[cfg(feature = "udp")]
pub mod udp {
//! UDP bindings for `tokio`.
//!
//! The main struct for UDP is the [`UdpSocket`], which represents a UDP socket.
//! Reading and writing to it can be done using futures, which return the
//! [`RecvDgram`] and [`SendDgram`] structs respectively.
//!
//! For convenience it's also possible to convert raw datagrams into higher-level
//! frames.
//!
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`RecvDgram`]: struct.RecvDgram.html
//! [`SendDgram`]: struct.SendDgram.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`framed`]: struct.UdpSocket.html#method.framed
pub use tokio_udp::{RecvDgram, SendDgram, UdpFramed, UdpSocket};
}
#[cfg(feature = "udp")]
pub use self::udp::{UdpFramed, UdpSocket};
#[cfg(feature = "udp")]
#[deprecated(note = "use `tokio::net::udp::RecvDgram` instead")]
#[doc(hidden)]
pub type RecvDgram<T> = self::udp::RecvDgram<T>;
#[cfg(feature = "udp")]
#[deprecated(note = "use `tokio::net::udp::SendDgram` instead")]
#[doc(hidden)]
pub type SendDgram<T> = self::udp::SendDgram<T>;
#[cfg(all(unix, feature = "uds"))]
pub mod unix {
//! Unix domain socket bindings for `tokio` (only available on unix systems).
pub use tokio_uds::{
ConnectFuture, Incoming, RecvDgram, SendDgram, UCred, UnixDatagram, UnixDatagramFramed,
UnixListener, UnixStream,
};
}
#[cfg(all(unix, feature = "uds"))]
pub use self::unix::{UnixDatagram, UnixDatagramFramed, UnixListener, UnixStream};
-55
View File
@@ -1,55 +0,0 @@
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
#[cfg(feature = "io")]
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
pub use util::{
FutureExt,
StreamExt,
};
pub use ::std::io::{
Read,
Write,
};
pub use futures::{
Future,
future,
Stream,
stream,
Sink,
IntoFuture,
Async,
AsyncSink,
Poll,
task,
};
#[cfg(feature = "async-await-preview")]
#[doc(inline)]
pub use tokio_async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt as StreamAsyncExt,
},
};
-149
View File
@@ -1,149 +0,0 @@
//! Event loop that drives Tokio I/O resources.
//!
//! This module contains [`Reactor`], which is the event loop that drives all
//! Tokio I/O resources. It is the reactor's job to receive events from the
//! operating system ([epoll], [kqueue], [IOCP], etc...) and forward them to
//! waiting tasks. It is the bridge between operating system and the futures
//! model.
//!
//! # Overview
//!
//! When using Tokio, all operations are asynchronous and represented by
//! futures. These futures, representing the application logic, are scheduled by
//! an executor (see [runtime model] for more details). Executors wait for
//! notifications before scheduling the future for execution time, i.e., nothing
//! happens until an event is received indicating that the task can make
//! progress.
//!
//! The reactor receives events from the operating system and notifies the
//! executor.
//!
//! Let's start with a basic example, establishing a TCP connection.
//!
//! ```rust
//! # extern crate tokio;
//! # fn dox() {
//! use tokio::prelude::*;
//! use tokio::net::TcpStream;
//!
//! let addr = "93.184.216.34:9243".parse().unwrap();
//!
//! let connect_future = TcpStream::connect(&addr);
//!
//! let task = connect_future
//! .and_then(|socket| {
//! println!("successfully connected");
//! Ok(())
//! })
//! .map_err(|e| println!("failed to connect; err={:?}", e));
//!
//! tokio::run(task);
//! # }
//! # fn main() {}
//! ```
//!
//! Establishing a TCP connection usually cannot be completed immediately.
//! [`TcpStream::connect`] does not block the current thread. Instead, it
//! returns a [future][connect-future] that resolves once the TCP connection has
//! been established. The connect future itself has no way of knowing when the
//! TCP connection has been established.
//!
//! Before returning the future, [`TcpStream::connect`] registers the socket
//! with a reactor. This registration process, handled by [`Registration`], is
//! what links the [`TcpStream`] with the [`Reactor`] instance. At this point,
//! the reactor starts listening for connection events from the operating system
//! for that socket.
//!
//! Once the connect future is passed to [`tokio::run`], it is spawned onto a
//! thread pool. The thread pool waits until it is notified that the connection
//! has completed.
//!
//! When the TCP connection is established, the reactor receives an event from
//! the operating system. It then notifies the thread pool, telling it that the
//! connect future can complete. At this point, the thread pool will schedule
//! the task to run on one of its worker threads. This results in the `and_then`
//! closure to get executed.
//!
//! ## Lazy registration
//!
//! Notice how the snippet above does not explicitly reference a reactor. When
//! [`TcpStream::connect`] is called, it registers the socket with a reactor,
//! but no reactor is specified. This works because the registration process
//! mentioned above is actually lazy. It doesn't *actually* happen in the
//! [`connect`] function. Instead, the registration is established the first
//! time that the task is polled (again, see [runtime model]).
//!
//! A reactor instance is automatically made available when using the Tokio
//! [runtime], which is done using [`tokio::run`]. The Tokio runtime's executor
//! sets a thread-local variable referencing the associated [`Reactor`] instance
//! and [`Handle::current`] (used by [`Registration`]) returns the reference.
//!
//! ## Implementation
//!
//! The reactor implementation uses [`mio`] to interface with the operating
//! system's event queue. A call to [`Reactor::poll`] results in a single
//! call to [`Poll::poll`] which in turn results in a single call to the
//! operating system's selector.
//!
//! The reactor maintains state for each registered I/O resource. This tracks
//! the executor task to notify when events are provided by the operating
//! system's selector. This state is stored in a `Sync` data structure and
//! referenced by [`Registration`]. When the [`Registration`] instance is
//! dropped, this state is cleaned up. Because the state is stored in a `Sync`
//! data structure, the [`Registration`] instance is able to be moved to other
//! threads.
//!
//! By default, a runtime's default reactor runs on a background thread. This
//! ensures that application code cannot significantly impact the reactor's
//! responsiveness.
//!
//! ## Integrating with the reactor
//!
//! Tokio comes with a number of I/O resources, like TCP and UDP sockets, that
//! automatically integrate with the reactor. However, library authors or
//! applications may wish to implement their own resources that are also backed
//! by the reactor.
//!
//! There are a couple of ways to do this.
//!
//! If the custom I/O resource implements [`mio::Evented`] and implements
//! [`std::io::Read`] and / or [`std::io::Write`], then [`PollEvented`] is the
//! most suited.
//!
//! Otherwise, [`Registration`] can be used directly. This provides the lowest
//! level primitive needed for integrating with the reactor: a stream of
//! readiness events.
//!
//! [`Reactor`]: struct.Reactor.html
//! [`Registration`]: struct.Registration.html
//! [runtime model]: https://tokio.rs/docs/getting-started/runtime-model/
//! [epoll]: http://man7.org/linux/man-pages/man7/epoll.7.html
//! [kqueue]: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
//! [IOCP]: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365198(v=vs.85).aspx
//! [`TcpStream::connect`]: ../net/struct.TcpStream.html#method.connect
//! [`connect`]: ../net/struct.TcpStream.html#method.connect
//! [connect-future]: ../net/struct.ConnectFuture.html
//! [`tokio::run`]: ../runtime/fn.run.html
//! [`TcpStream`]: ../net/struct.TcpStream.html
//! [runtime]: ../runtime
//! [`Handle::current`]: struct.Handle.html#method.current
//! [`mio`]: https://github.com/carllerche/mio
//! [`Reactor::poll`]: struct.Reactor.html#method.poll
//! [`Poll::poll`]: https://docs.rs/mio/0.6/mio/struct.Poll.html#method.poll
//! [`mio::Evented`]: https://docs.rs/mio/0.6/mio/trait.Evented.html
//! [`PollEvented`]: struct.PollEvented.html
//! [`std::io::Read`]: https://doc.rust-lang.org/std/io/trait.Read.html
//! [`std::io::Write`]: https://doc.rust-lang.org/std/io/trait.Write.html
pub use tokio_reactor::{
Reactor,
Handle,
Background,
Turn,
Registration,
PollEvented as PollEvented2,
};
mod poll_evented;
#[allow(deprecated)]
pub use self::poll_evented::PollEvented;
-539
View File
@@ -1,539 +0,0 @@
//! Readiness tracking streams, backing I/O objects.
//!
//! This module contains the core type which is used to back all I/O on object
//! in `tokio-core`. The `PollEvented` type is the implementation detail of
//! all I/O. Each `PollEvented` manages registration with a reactor,
//! acquisition of a token, and tracking of the readiness state on the
//! underlying I/O primitive.
#![allow(deprecated, warnings)]
use std::fmt;
use std::io::{self, Read, Write};
use std::sync::Mutex;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::{task, Async, Poll};
use mio::event::Evented;
use mio::Ready;
use tokio_io::{AsyncRead, AsyncWrite};
use reactor::{Handle, Registration};
#[deprecated(since = "0.1.2", note = "PollEvented2 instead")]
#[doc(hidden)]
pub struct PollEvented<E> {
io: E,
inner: Inner,
handle: Handle,
}
struct Inner {
registration: Mutex<Registration>,
/// Currently visible read readiness
read_readiness: AtomicUsize,
/// Currently visible write readiness
write_readiness: AtomicUsize,
}
impl<E: fmt::Debug> fmt::Debug for PollEvented<E> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("PollEvented")
.field("io", &self.io)
.finish()
}
}
impl<E> PollEvented<E> {
/// Creates a new readiness stream associated with the provided
/// `loop_handle` and for the given `source`.
pub fn new(io: E, handle: &Handle) -> io::Result<PollEvented<E>>
where E: Evented,
{
let registration = Registration::new();
registration.register(&io)?;
Ok(PollEvented {
io: io,
inner: Inner {
registration: Mutex::new(registration),
read_readiness: AtomicUsize::new(0),
write_readiness: AtomicUsize::new(0),
},
handle: handle.clone(),
})
}
/// Tests to see if this source is ready to be read from or not.
///
/// If this stream is not ready for a read then `Async::NotReady` will be
/// returned and the current task will be scheduled to receive a
/// notification when the stream is readable again. In other words, this
/// method is only safe to call from within the context of a future's task,
/// typically done in a `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::readable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_read(&mut self) -> Async<()> {
if self.poll_read2().is_ready() {
return ().into();
}
Async::NotReady
}
fn poll_read2(&self) -> Async<Ready> {
let r = self.inner.registration.lock().unwrap();
// Load the cached readiness
match self.inner.read_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_read_ready().unwrap() {
n |= ready2usize(ready);
self.inner.read_readiness.store(n, Relaxed);
}
return usize2ready(n).into();
}
}
let ready = match r.poll_read_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.read_readiness.store(ready2usize(ready), Relaxed);
ready.into()
}
/// Tests to see if this source is ready to be written to or not.
///
/// If this stream is not ready for a write then `Async::NotReady` will be returned
/// and the current task will be scheduled to receive a notification when
/// the stream is writable again. In other words, this method is only safe
/// to call from within the context of a future's task, typically done in a
/// `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::writable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_write(&mut self) -> Async<()> {
let r = self.inner.registration.lock().unwrap();
match self.inner.write_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_write_ready().unwrap() {
n |= ready2usize(ready);
self.inner.write_readiness.store(n, Relaxed);
}
return ().into();
}
}
let ready = match r.poll_write_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.write_readiness.store(ready2usize(ready), Relaxed);
().into()
}
/// Test to see whether this source fulfills any condition listed in `mask`
/// provided.
///
/// The `mask` given here is a mio `Ready` set of possible events. This can
/// contain any events like read/write but also platform-specific events
/// such as hup and error. The `mask` indicates events that are interested
/// in being ready.
///
/// If any event in `mask` is ready then it is returned through
/// `Async::Ready`. The `Ready` set returned is guaranteed to not be empty
/// and contains all events that are currently ready in the `mask` provided.
///
/// If no events are ready in the `mask` provided then the current task is
/// scheduled to receive a notification when any of them become ready. If
/// the `writable` event is contained within `mask` then this
/// `PollEvented`'s `write` task will be blocked and otherwise the `read`
/// task will be blocked. This is generally only relevant if you're working
/// with this `PollEvented` object on multiple tasks.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_ready(&mut self, mask: Ready) -> Async<Ready> {
let mut ret = Ready::empty();
if mask.is_empty() {
return ret.into();
}
if mask.is_writable() {
if self.poll_write().is_ready() {
ret = Ready::writable();
}
}
let mask = mask - Ready::writable();
if !mask.is_empty() {
if let Async::Ready(v) = self.poll_read2() {
ret |= v & mask;
}
}
if ret.is_empty() {
if mask.is_writable() {
let _ = self.need_write();
}
if mask.is_readable() {
let _ = self.need_read();
}
Async::NotReady
} else {
ret.into()
}
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer readable, but it needs to be.
///
/// This function, like `poll_read`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// readable, typically because a "would block" error was seen.
///
/// *All* readiness bits associated with this stream except the writable bit
/// will be reset when this method is called. The current task is then
/// scheduled to receive a notification whenever anything changes other than
/// the writable bit. Note that this typically just means the readable bit
/// is used here, but if you're using a custom I/O object for events like
/// hup/error this may also be relevant.
///
/// Note that it is also only valid to call this method if `poll_read`
/// previously indicated that the object is readable. That is, this function
/// must always be paired with calls to `poll_read` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_read(&mut self) -> io::Result<()> {
self.inner.read_readiness.store(0, Relaxed);
if self.poll_read().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer writable, but it needs to be.
///
/// This function, like `poll_write`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// writable, typically because a "would block" error was seen.
///
/// The flag indicating that this stream is writable is unset and the
/// current task is scheduled to receive a notification when the stream is
/// then again writable.
///
/// Note that it is also only valid to call this method if `poll_write`
/// previously indicated that the object is writable. That is, this function
/// must always be paired with calls to `poll_write` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_write(&mut self) -> io::Result<()> {
self.inner.write_readiness.store(0, Relaxed);
if self.poll_write().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Returns a reference to the event loop handle that this readiness stream
/// is associated with.
pub fn handle(&self) -> &Handle {
&self.handle
}
/// Returns a shared reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_ref(&self) -> &E {
&self.io
}
/// Returns a mutable reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_mut(&mut self) -> &mut E {
&mut self.io
}
/// Consumes the `PollEvented` and returns the underlying I/O object
pub fn into_inner(self) -> E {
self.io
}
/// Deregisters this source of events from the reactor core specified.
///
/// This method can optionally be called to unregister the underlying I/O
/// object with the event loop that the `handle` provided points to.
/// Typically this method is not required as this automatically happens when
/// `E` is dropped, but for some use cases the `E` object doesn't represent
/// an owned reference, so dropping it won't automatically unregister with
/// the event loop.
///
/// This consumes `self` as it will no longer provide events after the
/// method is called, and will likely return an error if this `PollEvented`
/// was created on a separate event loop from the `handle` specified.
pub fn deregister(&self) -> io::Result<()>
where E: Evented,
{
self.inner.registration.lock().unwrap()
.deregister(&self.io)
}
}
impl<E: Read> Read for PollEvented<E> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_read() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().read(buf);
if is_wouldblock(&r) {
self.need_read()?;
}
return r
}
}
impl<E: Write> Write for PollEvented<E> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().write(buf);
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
fn flush(&mut self) -> io::Result<()> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().flush();
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
}
impl<E: Read> AsyncRead for PollEvented<E> {
}
impl<E: Write> AsyncWrite for PollEvented<E> {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
fn is_wouldblock<T>(r: &io::Result<T>) -> bool {
match *r {
Ok(_) => false,
Err(ref e) => e.kind() == io::ErrorKind::WouldBlock,
}
}
const READ: usize = 1 << 0;
const WRITE: usize = 1 << 1;
fn ready2usize(ready: Ready) -> usize {
let mut bits = 0;
if ready.is_readable() {
bits |= READ;
}
if ready.is_writable() {
bits |= WRITE;
}
bits | platform::ready2usize(ready)
}
fn usize2ready(bits: usize) -> Ready {
let mut ready = Ready::empty();
if bits & READ != 0 {
ready.insert(Ready::readable());
}
if bits & WRITE != 0 {
ready.insert(Ready::writable());
}
ready | platform::usize2ready(bits)
}
#[cfg(unix)]
mod platform {
use mio::Ready;
use mio::unix::UnixReady;
const HUP: usize = 1 << 2;
const ERROR: usize = 1 << 3;
const AIO: usize = 1 << 4;
const LIO: usize = 1 << 5;
#[cfg(any(target_os = "dragonfly", target_os = "freebsd"))]
fn is_aio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_aio()
}
#[cfg(not(any(target_os = "dragonfly", target_os = "freebsd")))]
fn is_aio(_ready: &Ready) -> bool {
false
}
#[cfg(target_os = "freebsd")]
fn is_lio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_lio()
}
#[cfg(not(target_os = "freebsd"))]
fn is_lio(_ready: &Ready) -> bool {
false
}
pub fn ready2usize(ready: Ready) -> usize {
let ready = UnixReady::from(ready);
let mut bits = 0;
if is_aio(&ready) {
bits |= AIO;
}
if is_lio(&ready) {
bits |= LIO;
}
if ready.is_error() {
bits |= ERROR;
}
if ready.is_hup() {
bits |= HUP;
}
bits
}
#[cfg(any(target_os = "dragonfly", target_os = "freebsd", target_os = "ios",
target_os = "macos"))]
fn usize2ready_aio(ready: &mut UnixReady) {
ready.insert(UnixReady::aio());
}
#[cfg(not(any(target_os = "dragonfly",
target_os = "freebsd", target_os = "ios", target_os = "macos")))]
fn usize2ready_aio(_ready: &mut UnixReady) {
// aio not available here → empty
}
#[cfg(target_os = "freebsd")]
fn usize2ready_lio(ready: &mut UnixReady) {
ready.insert(UnixReady::lio());
}
#[cfg(not(target_os = "freebsd"))]
fn usize2ready_lio(_ready: &mut UnixReady) {
// lio not available here → empty
}
pub fn usize2ready(bits: usize) -> Ready {
let mut ready = UnixReady::from(Ready::empty());
if bits & AIO != 0 {
usize2ready_aio(&mut ready);
}
if bits & LIO != 0 {
usize2ready_lio(&mut ready);
}
if bits & HUP != 0 {
ready.insert(UnixReady::hup());
}
if bits & ERROR != 0 {
ready.insert(UnixReady::error());
}
ready.into()
}
}
#[cfg(windows)]
mod platform {
use mio::Ready;
pub fn all() -> Ready {
// No platform-specific Readinesses for Windows
Ready::empty()
}
pub fn hup() -> Ready {
Ready::empty()
}
pub fn ready2usize(_r: Ready) -> usize {
0
}
pub fn usize2ready(_r: usize) -> Ready {
Ready::empty()
}
}
-88
View File
@@ -1,88 +0,0 @@
use executor::current_thread::CurrentThread;
use runtime::current_thread::Runtime;
use tokio_reactor::Reactor;
use tokio_timer::clock::Clock;
use tokio_timer::timer::Timer;
use std::io;
/// Builds a Single-threaded runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// extern crate tokio;
/// extern crate tokio_timer;
///
/// use tokio::runtime::current_thread::Builder;
/// use tokio_timer::clock::Clock;
///
/// # pub fn main() {
/// // build Runtime
/// let runtime = Builder::new()
/// .clock(Clock::new())
/// .build();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
#[derive(Debug)]
pub struct Builder {
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
Builder {
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Create the configured `Runtime`.
pub fn build(&mut self) -> io::Result<Runtime> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new_with_now(reactor, self.clock.clone());
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor to generate some new stimuli for the
// futures to continue in their life.
let executor = CurrentThread::new_with_park(timer);
let runtime = Runtime::new2(
reactor_handle,
timer_handle,
self.clock.clone(),
executor);
Ok(runtime)
}
}
-107
View File
@@ -1,107 +0,0 @@
//! A runtime implementation that runs everything on the current thread.
//!
//! [`current_thread::Runtime`][rt] is similar to the primary
//! [`Runtime`][concurrent-rt] except that it runs all components on the current
//! thread instead of using a thread pool. This means that it is able to spawn
//! futures that do not implement `Send`.
//!
//! Same as the default [`Runtime`][concurrent-rt], the
//! [`current_thread::Runtime`][rt] includes:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! Note that [`current_thread::Runtime`][rt] does not implement `Send` itself
//! and cannot be safely moved to other threads.
//!
//! # Spawning from other threads
//!
//! While [`current_thread::Runtime`][rt] does not implement `Send` and cannot
//! safely be moved to other threads, it provides a `Handle` that can be sent
//! to other threads and allows to spawn new tasks from there.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//! use std::thread;
//!
//! # fn main() {
//! let mut runtime = Runtime::new().unwrap();
//! let handle = runtime.handle();
//!
//! thread::spawn(move || {
//! handle.spawn(future::ok(()));
//! }).join().unwrap();
//!
//! # /*
//! runtime.run().unwrap();
//! # */
//! # }
//! ```
//!
//! # Examples
//!
//! Creating a new `Runtime` and running a future `f` until its completion and
//! returning its result.
//!
//! ```
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//!
//! let mut runtime = Runtime::new().unwrap();
//!
//! // Use the runtime...
//! // runtime.block_on(f); // where f is a future
//! ```
//!
//! [rt]: struct.Runtime.html
//! [concurrent-rt]: ../struct.Runtime.html
//! [chan]: https://docs.rs/futures/0.1/futures/sync/mpsc/fn.channel.html
//! [reactor]: ../../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../../timer/index.html
mod builder;
mod runtime;
pub use self::builder::Builder;
pub use self::runtime::{Runtime, Handle};
pub use tokio_current_thread::spawn;
pub use tokio_current_thread::TaskExecutor;
use futures::Future;
/// Run the provided future to completion using a runtime running on the current thread.
///
/// This first creates a new [`Runtime`], and calls [`Runtime::block_on`] with the provided future,
/// which blocks the current thread until the provided future completes. It then calls
/// [`Runtime::run`] to wait for any other spawned futures to resolve.
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where
F: Future,
{
let mut r = Runtime::new().expect("failed to start runtime on current thread");
let v = r.block_on(future)?;
r.run().expect("failed to resolve remaining futures");
Ok(v)
}
/// Start a current-thread runtime using the supplied future to bootstrap execution.
///
/// # Panics
///
/// This function panics if called from the context of an executor.
pub fn run<F>(future: F)
where
F: Future<Item = (), Error = ()> + 'static,
{
let mut r = Runtime::new().expect("failed to start runtime on current thread");
r.spawn(future);
r.run().expect("failed to resolve remaining futures");
}
-238
View File
@@ -1,238 +0,0 @@
use tokio_current_thread::{self as current_thread, CurrentThread};
use tokio_current_thread::Handle as ExecutorHandle;
use runtime::current_thread::Builder;
use tokio_reactor::{self, Reactor};
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
use tokio_executor;
use futures::{future, Future};
use std::fmt;
use std::error::Error;
use std::io;
/// Single-threaded runtime provides a way to start reactor
/// and executor on the current thread.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
#[derive(Debug)]
pub struct Runtime {
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>,
}
/// Handle to spawn a future on the corresponding `CurrentThread` runtime instance
#[derive(Debug, Clone)]
pub struct Handle(ExecutorHandle);
impl Handle {
/// Spawn a future onto the `CurrentThread` runtime instance corresponding to this handle
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the `CurrentThread`
/// instance of the `Handle` does not exist anymore.
pub fn spawn<F>(&self, future: F) -> Result<(), tokio_executor::SpawnError>
where F: Future<Item = (), Error = ()> + Send + 'static {
self.0.spawn(future)
}
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
pub fn status(&self) -> Result<(), tokio_executor::SpawnError> {
self.0.status()
}
}
impl<T> future::Executor<T> for Handle
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
if let Err(e) = self.status() {
let kind = if e.is_at_capacity() {
future::ExecuteErrorKind::NoCapacity
} else {
future::ExecuteErrorKind::Shutdown
};
return Err(future::ExecuteError::new(kind, future));
}
let _ = self.spawn(future);
Ok(())
}
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
inner: current_thread::RunError,
}
impl fmt::Display for RunError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.inner)
}
}
impl Error for RunError {
fn description(&self) -> &str {
self.inner.description()
}
// FIXME(taiki-e): When the minimum support version of tokio reaches Rust 1.30,
// replace this with Error::source.
#[allow(deprecated)]
fn cause(&self) -> Option<&Error> {
self.inner.cause()
}
}
impl Runtime {
/// Returns a new runtime initialized with default configuration values.
pub fn new() -> io::Result<Runtime> {
Builder::new().build()
}
pub(super) fn new2(
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>) -> Runtime
{
Runtime {
reactor_handle,
timer_handle,
clock,
executor,
}
}
/// Get a new handle to spawn futures on the single-threaded Tokio runtime
///
/// Different to the runtime itself, the handle can be sent to different
/// threads.
pub fn handle(&self) -> Handle {
Handle(self.executor.handle().clone())
}
/// Spawn a future onto the single-threaded Tokio runtime.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::current_thread::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("running on the runtime");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.spawn(future);
self
}
/// Runs the provided future, blocking the current thread until the future
/// completes.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed. Once the function returns, any uncompleted futures
/// remain pending in the `Runtime` instance. These futures will not run
/// until `block_on` or `run` is called again.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution by calling `block_on` or `run`.
pub fn block_on<F>(&mut self, f: F) -> Result<F::Item, F::Error>
where F: Future
{
self.enter(|executor| {
// Run the provided future
let ret = executor.block_on(f);
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
})
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.enter(|executor| executor.run())
.map_err(|e| RunError {
inner: e,
})
}
fn enter<F, R>(&mut self, f: F) -> R
where F: FnOnce(&mut current_thread::Entered<Timer<Reactor>>) -> R
{
let Runtime {
ref reactor_handle,
ref timer_handle,
ref clock,
ref mut executor,
..
} = *self;
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure
// and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
clock::with_default(clock, enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the
// current single-threaded executor when used. This is a trick,
// because we need two mutable references to the executor (one
// to run the provided future, another to install as the default
// one). We use the fake one here as the default one.
let mut default_executor = current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
f(&mut executor)
})
})
})
})
}
}
-125
View File
@@ -1,125 +0,0 @@
//! A batteries included runtime for applications using Tokio.
//!
//! Applications using Tokio require some runtime support in order to work:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
//!
//! [`Runtime`] bundles all of these various runtime components into a single
//! handle that can be started and shutdown together, eliminating the necessary
//! boilerplate to run a Tokio application.
//!
//! Most applications wont need to use [`Runtime`] directly. Instead, they will
//! use the [`run`] function, which uses [`Runtime`] under the hood.
//!
//! Creating a [`Runtime`] does the following:
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//! * Run an instance of [`Timer`] **per** thread pool worker thread.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! A timer per thread pool worker thread is used to minimize the amount of
//! synchronization that is required for working with the timer.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
//! "seed" the application, blocking the thread until the runtime becomes
//! [idle].
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! tokio::run(server);
//! # }
//! # pub fn main() {}
//! ```
//!
//! In this function, the `run` function blocks until the runtime becomes idle.
//! See [`shutdown_on_idle`][idle] for more shutdown details.
//!
//! From within the context of the runtime, additional tasks are spawned using
//! the [`tokio::spawn`] function. Futures spawned using this function will be
//! executed on the same thread pool used by the [`Runtime`].
//!
//! A [`Runtime`] instance can also be used directly.
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::runtime::Runtime;
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! // Create the runtime
//! let mut rt = Runtime::new().unwrap();
//!
//! // Spawn the server task
//! rt.spawn(server);
//!
//! // Wait until the runtime becomes idle and shut it down.
//! rt.shutdown_on_idle()
//! .wait().unwrap();
//! # }
//! # pub fn main() {}
//! ```
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../timer/index.html
//! [`Runtime`]: struct.Runtime.html
//! [`Reactor`]: ../reactor/struct.Reactor.html
//! [`ThreadPool`]: https://docs.rs/tokio-threadpool/0.1/tokio_threadpool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
//! [`Timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/timer/struct.Timer.html
pub mod current_thread;
mod threadpool;
pub use self::threadpool::{
Builder,
Runtime,
Shutdown,
TaskExecutor,
run,
};
-368
View File
@@ -1,368 +0,0 @@
use super::{Inner, Runtime};
use reactor::Reactor;
use std::io;
use std::sync::Mutex;
use std::time::Duration;
use num_cpus;
use tokio_reactor;
use tokio_threadpool::Builder as ThreadPoolBuilder;
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
/// Builds Tokio Runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// extern crate tokio;
/// extern crate tokio_timer;
///
/// use std::time::Duration;
///
/// use tokio::runtime::Builder;
/// use tokio_timer::clock::Clock;
///
/// fn main() {
/// // build Runtime
/// let mut runtime = Builder::new()
/// .blocking_threads(4)
/// .clock(Clock::system())
/// .core_threads(4)
/// .keep_alive(Some(Duration::from_secs(60)))
/// .name_prefix("my-custom-name-")
/// .stack_size(3 * 1024 * 1024)
/// .build()
/// .unwrap();
///
/// // use runtime ...
/// }
/// ```
#[derive(Debug)]
pub struct Builder {
/// Thread pool specific builder
threadpool_builder: ThreadPoolBuilder,
/// The number of worker threads
core_threads: usize,
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
let core_threads = num_cpus::get().max(1);
let mut threadpool_builder = ThreadPoolBuilder::new();
threadpool_builder.name_prefix("tokio-runtime-worker-");
threadpool_builder.pool_size(core_threads);
Builder {
threadpool_builder,
core_threads,
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Set builder to set up the thread pool instance.
#[deprecated(
since="0.1.9",
note="use the `core_threads`, `blocking_threads`, `name_prefix`, \
`keep_alive`, and `stack_size` functions on `runtime::Builder`, \
instead")]
#[doc(hidden)]
pub fn threadpool_builder(&mut self, val: ThreadPoolBuilder) -> &mut Self {
self.threadpool_builder = val;
self
}
/// Set the maximum number of worker threads for the `Runtime`'s thread pool.
///
/// This must be a number between 1 and 32,768 though it is advised to keep
/// this value on the smaller side.
///
/// The default value is the number of cores available to the system.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .core_threads(4)
/// .build()
/// .unwrap();
/// # }
/// ```
pub fn core_threads(&mut self, val: usize) -> &mut Self {
self.core_threads = val;
self.threadpool_builder.pool_size(val);
self
}
/// Set the maximum number of concurrent blocking sections in the `Runtime`'s
/// thread pool.
///
/// When the maximum concurrent `blocking` calls is reached, any further
/// calls to `blocking` will return `NotReady` and the task is notified once
/// previously in-flight calls to `blocking` return.
///
/// This must be a number between 1 and 32,768 though it is advised to keep
/// this value on the smaller side.
///
/// The default value is 100.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .blocking_threads(200)
/// .build();
/// # }
/// ```
pub fn blocking_threads(&mut self, val: usize) -> &mut Self {
self.threadpool_builder.max_blocking(val);
self
}
/// Set the worker thread keep alive duration for threads in the `Runtime`'s
/// thread pool.
///
/// If set, a worker thread will wait for up to the specified duration for
/// work, at which point the thread will shutdown. When work becomes
/// available, a new thread will eventually be spawned to replace the one
/// that shut down.
///
/// When the value is `None`, the thread will wait for work forever.
///
/// The default value is `None`.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
/// use std::time::Duration;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .keep_alive(Some(Duration::from_secs(30)))
/// .build();
/// # }
/// ```
pub fn keep_alive(&mut self, val: Option<Duration>) -> &mut Self {
self.threadpool_builder.keep_alive(val);
self
}
/// Set name prefix of threads spawned by the `Runtime`'s thread pool.
///
/// Thread name prefix is used for generating thread names. For example, if
/// prefix is `my-pool-`, then threads in the pool will get names like
/// `my-pool-1` etc.
///
/// The default prefix is "tokio-runtime-worker-".
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .name_prefix("my-pool-")
/// .build();
/// # }
/// ```
pub fn name_prefix<S: Into<String>>(&mut self, val: S) -> &mut Self {
self.threadpool_builder.name_prefix(val);
self
}
/// Set the stack size (in bytes) for worker threads.
///
/// The actual stack size may be greater than this value if the platform
/// specifies minimal stack size.
///
/// The default stack size for spawned threads is 2 MiB, though this
/// particular stack size is subject to change in the future.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .stack_size(32 * 1024)
/// .build();
/// # }
/// ```
pub fn stack_size(&mut self, val: usize) -> &mut Self {
self.threadpool_builder.stack_size(val);
self
}
/// Execute function `f` after each thread is started but before it starts
/// doing work.
///
/// This is intended for bookkeeping and monitoring use cases.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let thread_pool = runtime::Builder::new()
/// .after_start(|| {
/// println!("thread started");
/// })
/// .build();
/// # }
/// ```
pub fn after_start<F>(&mut self, f: F) -> &mut Self
where F: Fn() + Send + Sync + 'static
{
self.threadpool_builder.after_start(f);
self
}
/// Execute function `f` before each thread stops.
///
/// This is intended for bookkeeping and monitoring use cases.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let thread_pool = runtime::Builder::new()
/// .before_stop(|| {
/// println!("thread stopping");
/// })
/// .build();
/// # }
/// ```
pub fn before_stop<F>(&mut self, f: F) -> &mut Self
where F: Fn() + Send + Sync + 'static
{
self.threadpool_builder.before_stop(f);
self
}
/// Create the configured `Runtime`.
///
/// The returned `ThreadPool` instance is ready to spawn tasks.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::runtime::Builder;
/// # pub fn main() {
/// let runtime = Builder::new().build().unwrap();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
pub fn build(&mut self) -> io::Result<Runtime> {
// TODO(stjepang): Once we remove the `threadpool_builder` method, remove this line too.
self.threadpool_builder.pool_size(self.core_threads);
let mut reactor_handles = Vec::new();
let mut timer_handles = Vec::new();
let mut timers = Vec::new();
for _ in 0..self.core_threads {
// Create a new reactor.
let reactor = Reactor::new()?;
reactor_handles.push(reactor.handle());
// Create a new timer.
let timer = Timer::new_with_now(reactor, self.clock.clone());
timer_handles.push(timer.handle());
timers.push(Mutex::new(Some(timer)));
}
// Get a handle to the clock for the runtime.
let clock = self.clock.clone();
let pool = self.threadpool_builder
.around_worker(move |w, enter| {
let index = w.id().to_usize();
tokio_reactor::with_default(&reactor_handles[index], enter, |enter| {
clock::with_default(&clock, enter, |enter| {
timer::with_default(&timer_handles[index], enter, |_| {
w.run();
});
})
});
})
.custom_park(move |worker_id| {
let index = worker_id.to_usize();
timers[index]
.lock()
.unwrap()
.take()
.unwrap()
})
.build();
// To support deprecated `reactor()` function
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
Ok(Runtime {
inner: Some(Inner {
reactor_handle,
reactor: Mutex::new(Some(reactor)),
pool,
}),
})
}
}
-395
View File
@@ -1,395 +0,0 @@
mod builder;
mod shutdown;
mod task_executor;
pub use self::builder::Builder;
pub use self::shutdown::Shutdown;
pub use self::task_executor::TaskExecutor;
use reactor::{Handle, Reactor};
use std::io;
use std::sync::Mutex;
use tokio_executor::enter;
use tokio_threadpool as threadpool;
use futures;
use futures::future::Future;
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// Instances of `Runtime` can be created using [`new`] or [`Builder`]. However,
/// most users will use [`tokio::run`], which uses a `Runtime` internally.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
/// [`new`]: #method.new
/// [`Builder`]: struct.Builder.html
/// [`tokio::run`]: fn.run.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
#[derive(Debug)]
struct Inner {
/// A handle to the reactor in the background thread.
reactor_handle: Handle,
// TODO: This should go away in 0.2
reactor: Mutex<Option<Reactor>>,
/// Task execution pool.
pool: threadpool::ThreadPool,
}
// ===== impl Runtime =====
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// This function is used to bootstrap the execution of a Tokio application. It
/// does the following:
///
/// * Start the Tokio runtime using a default configuration.
/// * Spawn the given future onto the thread pool.
/// * Block the current thread until the runtime shuts down.
///
/// Note that the function will not return immediately once `future` has
/// completed. Instead it waits for the entire runtime to become idle.
///
/// See the [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if called from the context of an executor.
///
/// [mod]: ../index.html
pub fn run<F>(future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
// Check enter before creating a new Runtime...
let mut entered = enter().expect("nested tokio::run");
let mut runtime = Runtime::new().expect("failed to start new Runtime");
runtime.spawn(future);
entered
.block_on(runtime.shutdown_on_idle())
.expect("shutdown cannot error")
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// This results in a reactor, thread pool, and timer being initialized. The
/// thread pool will not spawn any worker threads until it needs to, i.e.
/// tasks are scheduled to run.
///
/// Most users will not need to call this function directly, instead they
/// will use [`tokio::run`](fn.run.html).
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// Creating a new `Runtime` with default configuration values.
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
Builder::new().build()
}
#[deprecated(since = "0.1.5", note = "use `reactor` instead")]
#[doc(hidden)]
pub fn handle(&self) -> &Handle {
#[allow(deprecated)]
self.reactor()
}
/// Return a reference to the reactor handle for this runtime instance.
///
/// The returned handle reference can be cloned in order to get an owned
/// value of the handle. This handle can be used to initialize I/O resources
/// (like TCP or UDP sockets) that will not be used on the runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let reactor_handle = rt.reactor().clone();
///
/// // use `reactor_handle`
/// ```
#[deprecated(since = "0.1.11", note = "there is now a reactor per worker thread")]
pub fn reactor(&self) -> &Handle {
let mut reactor = self.inner().reactor.lock().unwrap();
if let Some(reactor) = reactor.take() {
if let Ok(background) = reactor.background() {
background.forget();
}
}
&self.inner().reactor_handle
}
/// Return a handle to the runtime's executor.
///
/// The returned handle can be used to spawn tasks that run on this runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let executor_handle = rt.executor();
///
/// // use `executor_handle`
/// ```
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
}
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner_mut().pool.sender().spawn(future).unwrap();
self
}
/// Run a future to completion on the Tokio runtime.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, and yielding its resolved result. Any tasks or timers which
/// the future spawns internally will be executed on the runtime.
///
/// This method should not be called from an asynchronous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on<F, R, E>(&mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let mut entered = enter().expect("nested block_on");
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
entered.block_on(rx).unwrap()
}
/// Run a future to completion on the Tokio runtime, then wait for all
/// background futures to complete too.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, waiting for background futures to complete, and yielding
/// its resolved result. Any tasks or timers which the future spawns
/// internally will be executed on the runtime and waited for completion.
///
/// This method should not be called from an asynchronous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on_all<F, R, E>(mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let mut entered = enter().expect("nested block_on_all");
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
let block = rx
.map_err(|_| unreachable!())
.and_then(move |r| {
self.shutdown_on_idle()
.map(move |()| r)
});
entered.block_on(block).unwrap()
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function can be used to perform a graceful shutdown of the runtime.
///
/// The runtime enters an idle state once **all** of the following occur.
///
/// * The thread pool has no tasks to execute, i.e., all tasks that were
/// spawned have completed.
/// * The reactor is not managing any I/O resources.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_on_idle()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = inner.pool.shutdown_on_idle();
Shutdown { inner }
}
/// Signals the runtime to shutdown immediately.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function will forcibly shutdown the runtime, causing any
/// in-progress work to become canceled. The shutdown steps are:
///
/// * Drain any scheduled work queues.
/// * Drop any futures that have not yet completed.
/// * Drop the reactor.
///
/// Once the reactor has dropped, any outstanding I/O resources bound to
/// that reactor will no longer function. Calling any method on them will
/// result in an error.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
Shutdown::shutdown_now(inner)
}
fn inner(&self) -> &Inner {
self.inner.as_ref().unwrap()
}
fn inner_mut(&mut self) -> &mut Inner {
self.inner.as_mut().unwrap()
}
}
impl Drop for Runtime {
fn drop(&mut self) {
if let Some(inner) = self.inner.take() {
let shutdown = Shutdown::shutdown_now(inner);
let _ = shutdown.wait();
}
}
}
-36
View File
@@ -1,36 +0,0 @@
use super::Inner;
use tokio_threadpool as threadpool;
use std::fmt;
use futures::{Future, Poll};
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
pub(super) inner: threadpool::Shutdown,
}
impl Shutdown {
pub(super) fn shutdown_now(inner: Inner) -> Self {
let inner = inner.pool.shutdown_now();
Shutdown { inner }
}
}
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
-75
View File
@@ -1,75 +0,0 @@
use tokio_threadpool::Sender;
use futures::future::{self, Future};
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
pub(super) inner: Sender,
}
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
}
-16
View File
@@ -1,16 +0,0 @@
//! Future-aware synchronization
//!
//! This module is enabled with the **`sync`** feature flag.
//!
//! Tasks sometimes need to communicate with each other. This module contains
//! two basic abstractions for doing so:
//!
//! - [oneshot](oneshot/index.html), a way of sending a single value
//! from one task to another.
//! - [mpsc](mpsc/index.html), a multi-producer, single-consumer channel for
//! sending values between tasks.
pub use tokio_sync::{
mpsc,
oneshot,
};
-102
View File
@@ -1,102 +0,0 @@
//! Utilities for tracking time.
//!
//! This module provides a number of types for executing code after a set period
//! of time.
//!
//! * [`Delay`][Delay] is a future that does no work and completes at a specific `Instant`
//! in time.
//!
//! * [`Interval`][Interval] is a stream yielding a value at a fixed period. It
//! is initialized with a `Duration` and repeatedly yields each time the
//! duration elapses.
//!
//! * [`Timeout`][Timeout]: Wraps a future or stream, setting an upper bound to the
//! amount of time it is allowed to execute. If the future or stream does not
//! complete in time, then it is canceled and an error is returned.
//!
//! * [`DelayQueue`]: A queue where items are returned once the requested delay
//! has expired.
//!
//! These types are sufficient for handling a large number of scenarios
//! involving time.
//!
//! These types must be used from within the context of the
//! [`Runtime`][runtime] or a timer context must be setup explicitly. See the
//! [`tokio-timer`][tokio-timer] crate for more details on how to setup a timer
//! context.
//!
//! # Examples
//!
//! Wait 100ms and print "Hello World!"
//!
//! ```
//! use tokio::prelude::*;
//! use tokio::timer::Delay;
//!
//! use std::time::{Duration, Instant};
//!
//! let when = Instant::now() + Duration::from_millis(100);
//!
//! tokio::run({
//! Delay::new(when)
//! .map_err(|e| panic!("timer failed; err={:?}", e))
//! .and_then(|_| {
//! println!("Hello world!");
//! Ok(())
//! })
//! })
//! ```
//!
//! Require that an operation takes no more than 300ms. Note that this uses the
//! [`timeout`][ext] function on the [`FutureExt`][ext] trait. This trait is
//! included in the prelude.
//!
//! ```
//! # extern crate futures;
//! # extern crate tokio;
//! use tokio::prelude::*;
//!
//! use std::time::{Duration, Instant};
//!
//! fn long_op() -> Box<Future<Item = (), Error = ()> + Send> {
//! // ...
//! # Box::new(futures::future::ok(()))
//! }
//!
//! # fn main() {
//! tokio::run({
//! long_op()
//! .timeout(Duration::from_millis(300))
//! .map_err(|e| {
//! println!("operation timed out");
//! })
//! })
//! # }
//! ```
//!
//! [runtime]: ../runtime/struct.Runtime.html
//! [tokio-timer]: https://docs.rs/tokio-timer
//! [ext]: ../util/trait.FutureExt.html#method.timeout
//! [Timeout]: struct.Timeout.html
//! [Delay]: struct.Delay.html
//! [Interval]: struct.Interval.html
//! [`DelayQueue`]: struct.DelayQueue.html
pub use tokio_timer::{
delay_queue,
DelayQueue,
Error,
Interval,
Delay,
Timeout,
timeout,
};
#[deprecated(since = "0.1.8", note = "use Timeout instead")]
#[allow(deprecated)]
#[doc(hidden)]
pub type Deadline<T> = ::tokio_timer::Deadline<T>;
#[deprecated(since = "0.1.8", note = "use Timeout instead")]
#[allow(deprecated)]
#[doc(hidden)]
pub type DeadlineError<T> = ::tokio_timer::DeadlineError<T>;
-80
View File
@@ -1,80 +0,0 @@
use futures::{Async, Poll, Stream, Sink, StartSend};
/// A stream combinator which combines the yields the current item
/// plus its count starting from 0.
///
/// This structure is produced by the `Stream::enumerate` method.
#[derive(Debug)]
#[must_use = "Does nothing unless polled"]
pub struct Enumerate<T> {
inner: T,
count: usize,
}
impl<T> Enumerate<T> {
pub(crate) fn new(stream: T) -> Self {
Self { inner: stream, count: 0 }
}
/// Acquires a reference to the underlying stream that this combinator is
/// pulling from.
pub fn get_ref(&self) -> &T {
&self.inner
}
/// Acquires a mutable reference to the underlying stream that this
/// combinator is pulling from.
///
/// Note that care must be taken to avoid tampering with the state of the
/// stream which may otherwise confuse this combinator.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
/// Consumes this combinator, returning the underlying stream.
///
/// Note that this may discard intermediate state of this combinator, so
/// care should be taken to avoid losing resources when this is called.
pub fn into_inner(self) -> T {
self.inner
}
}
impl<T> Stream for Enumerate<T>
where
T: Stream,
{
type Item = (usize, T::Item);
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, T::Error> {
match try_ready!(self.inner.poll()) {
Some(item) => {
let ret = Some((self.count, item));
self.count += 1;
Ok(Async::Ready(ret))
}
None => return Ok(Async::Ready(None)),
}
}
}
// Forwarding impl of Sink from the underlying stream
impl<T> Sink for Enumerate<T>
where T: Sink
{
type SinkItem = T::SinkItem;
type SinkError = T::SinkError;
fn start_send(&mut self, item: T::SinkItem) -> StartSend<T::SinkItem, T::SinkError> {
self.inner.start_send(item)
}
fn poll_complete(&mut self) -> Poll<(), T::SinkError> {
self.inner.poll_complete()
}
fn close(&mut self) -> Poll<(), T::SinkError> {
self.inner.close()
}
}
-93
View File
@@ -1,93 +0,0 @@
#[cfg(feature = "timer")]
#[allow(deprecated)]
use tokio_timer::Deadline;
#[cfg(feature = "timer")]
use tokio_timer::Timeout;
use futures::Future;
#[cfg(feature = "timer")]
use std::time::{Instant, Duration};
/// An extension trait for `Future` that provides a variety of convenient
/// combinator functions.
///
/// Currently, there only is a [`timeout`] function, but this will increase
/// over time.
///
/// Users are not expected to implement this trait. All types that implement
/// `Future` already implement `FutureExt`.
///
/// This trait can be imported directly or via the Tokio prelude: `use
/// tokio::prelude::*`.
///
/// [`timeout`]: #method.timeout
pub trait FutureExt: Future {
/// Creates a new future which allows `self` until `timeout`.
///
/// This combinator creates a new future which wraps the receiving future
/// with a timeout. The returned future is allowed to execute until it
/// completes or `timeout` has elapsed, whichever happens first.
///
/// If the future completes before `timeout` then the future will resolve
/// with that item. Otherwise the future will resolve to an error.
///
/// The future is guaranteed to be polled at least once, even if `timeout`
/// is set to zero.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// use tokio::prelude::*;
/// use std::time::Duration;
/// # use futures::future::{self, FutureResult};
///
/// # fn long_future() -> FutureResult<(), ()> {
/// # future::ok(())
/// # }
/// #
/// # fn main() {
/// let future = long_future()
/// .timeout(Duration::from_secs(1))
/// .map_err(|e| println!("error = {:?}", e));
///
/// tokio::run(future);
/// # }
/// ```
#[cfg(feature = "timer")]
fn timeout(self, timeout: Duration) -> Timeout<Self>
where Self: Sized,
{
Timeout::new(self, timeout)
}
#[cfg(feature = "timer")]
#[deprecated(since = "0.1.8", note = "use `timeout` instead")]
#[allow(deprecated)]
#[doc(hidden)]
fn deadline(self, deadline: Instant) -> Deadline<Self>
where Self: Sized,
{
Deadline::new(self, deadline)
}
}
impl<T: ?Sized> FutureExt for T where T: Future {}
#[cfg(test)]
mod test {
use super::*;
use prelude::future;
#[cfg(feature = "timer")]
#[test]
fn timeout_polls_at_least_once() {
let base_future = future::result::<(), ()>(Ok(()));
let timeouted_future = base_future.timeout(Duration::new(0, 0));
assert!(timeouted_future.wait().is_ok());
}
}
-15
View File
@@ -1,15 +0,0 @@
//! Utilities for working with Tokio.
//!
//! This module contains utilities that are useful for working with Tokio.
//! Currently, this only includes [`FutureExt`] and [`StreamExt`], but this
//! may grow over time.
//!
//! [`FutureExt`]: trait.FutureExt.html
//! [`StreamExt`]: trait.StreamExt.html
mod future;
mod stream;
mod enumerate;
pub use self::future::FutureExt;
pub use self::stream::StreamExt;
-95
View File
@@ -1,95 +0,0 @@
#[cfg(feature = "timer")]
use tokio_timer::{
throttle::Throttle,
Timeout,
};
use futures::Stream;
#[cfg(feature = "timer")]
use std::time::Duration;
pub use util::enumerate::Enumerate;
/// An extension trait for `Stream` that provides a variety of convenient
/// combinator functions.
///
/// Currently, there only is a [`timeout`] function, but this will increase
/// over time.
///
/// Users are not expected to implement this trait. All types that implement
/// `Stream` already implement `StreamExt`.
///
/// This trait can be imported directly or via the Tokio prelude: `use
/// tokio::prelude::*`.
///
/// [`timeout`]: #method.timeout
pub trait StreamExt: Stream {
/// Throttle down the stream by enforcing a fixed delay between items.
///
/// Errors are also delayed.
#[cfg(feature = "timer")]
fn throttle(self, duration: Duration) -> Throttle<Self>
where Self: Sized
{
Throttle::new(self, duration)
}
/// Creates a new stream which gives the current iteration count as well
/// as the next value.
///
/// The stream returned yields pairs `(i, val)`, where `i` is the
/// current index of iteration and `val` is the value returned by the
/// iterator.
///
/// # Overflow Behavior
///
/// The method does no guarding against overflows, so counting elements of
/// an iterator with more than [`std::usize::MAX`] elements either produces the
/// wrong result or panics.
fn enumerate(self) -> Enumerate<Self>
where Self: Sized,
{
Enumerate::new(self)
}
/// Creates a new stream which allows `self` until `timeout`.
///
/// This combinator creates a new stream which wraps the receiving stream
/// with a timeout. For each item, the returned stream is allowed to execute
/// until it completes or `timeout` has elapsed, whichever happens first.
///
/// If an item completes before `timeout` then the stream will yield
/// with that item. Otherwise the stream will yield to an error.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// use tokio::prelude::*;
/// use std::time::Duration;
/// # use futures::future::{self, FutureResult};
///
/// # fn long_future() -> FutureResult<(), ()> {
/// # future::ok(())
/// # }
/// #
/// # fn main() {
/// let stream = long_future()
/// .into_stream()
/// .timeout(Duration::from_secs(1))
/// .for_each(|i| future::ok(println!("item = {:?}", i)))
/// .map_err(|e| println!("error = {:?}", e));
///
/// tokio::run(stream);
/// # }
/// ```
#[cfg(feature = "timer")]
fn timeout(self, timeout: Duration) -> Timeout<Self>
where Self: Sized,
{
Timeout::new(self, timeout)
}
}
impl<T: ?Sized> StreamExt for T where T: Stream {}
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "tests-build"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[features]
full = ["tokio/full"]
[dependencies]
tokio = { path = "../tokio", optional = true }
[dev-dependencies]
trybuild = "1.0"
+2
View File
@@ -0,0 +1,2 @@
Tests the various combination of feature flags. This is broken out to a separate
crate to work around limitations with cargo features.
+2
View File
@@ -0,0 +1,2 @@
#[cfg(feature = "tokio")]
pub use tokio;
@@ -0,0 +1,25 @@
use tests_build::tokio;
#[tokio::main]
fn main_is_not_async() {}
#[tokio::main(foo)]
async fn main_attr_has_unknown_args() {}
#[tokio::main(threadpool::bar)]
async fn main_attr_has_path_args() {}
#[tokio::test]
fn test_is_not_async() {}
#[tokio::test]
async fn test_fn_has_args(_x: u8) {}
#[tokio::test(foo)]
async fn test_attr_has_args() {}
#[tokio::test]
#[test]
async fn test_has_second_test_attr() {}
fn main() {}
@@ -0,0 +1,41 @@
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:4:1
|
4 | fn main_is_not_async() {}
| ^^
error: Unknown attribute foo is specified; expected `basic_scheduler` or `threaded_scheduler`
--> $DIR/macros_invalid_input.rs:6:15
|
6 | #[tokio::main(foo)]
| ^^^
error: Must have specified ident
--> $DIR/macros_invalid_input.rs:9:15
|
9 | #[tokio::main(threadpool::bar)]
| ^^^^^^^^^^^^^^^
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:13:1
|
13 | fn test_is_not_async() {}
| ^^
error: the test function cannot accept arguments
--> $DIR/macros_invalid_input.rs:16:27
|
16 | async fn test_fn_has_args(_x: u8) {}
| ^^^^^^
error: Unknown attribute foo is specified; expected `basic_scheduler` or `threaded_scheduler`
--> $DIR/macros_invalid_input.rs:18:15
|
18 | #[tokio::test(foo)]
| ^^^
error: second test attribute is supplied
--> $DIR/macros_invalid_input.rs:22:1
|
22 | #[test]
| ^^^^^^^
+9
View File
@@ -0,0 +1,9 @@
#[test]
fn compile_fail() {
let t = trybuild::TestCases::new();
#[cfg(feature = "full")]
t.compile_fail("tests/fail/macros_invalid_input.rs");
drop(t);
}
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "tests-integration"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[dependencies]
tokio = { path = "../tokio", features = ["full"] }
doc-comment = "0.3.1"
[dev-dependencies]
tokio-test = { path = "../tokio-test" }
futures = { version = "0.3.0", features = ["async-await"] }
+1
View File
@@ -0,0 +1 @@
Tests that require additional components than just the `tokio` crate.
+20
View File
@@ -0,0 +1,20 @@
//! A cat-like utility that can be used as a subprocess to test I/O
//! stream communication.
use std::io;
use std::io::Write;
fn main() {
let stdin = io::stdin();
let mut stdout = io::stdout();
let mut line = String::new();
loop {
line.clear();
stdin.read_line(&mut line).unwrap();
if line.is_empty() {
break;
}
stdout.write_all(line.as_bytes()).unwrap();
}
stdout.flush().unwrap();
}
+4
View File
@@ -0,0 +1,4 @@
use doc_comment::doc_comment;
// #[doc = include_str!("../../README.md")]
doc_comment!(include_str!("../../README.md"));
+126
View File
@@ -0,0 +1,126 @@
#![warn(rust_2018_idioms)]
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::process::{Child, Command};
use tokio_test::assert_ok;
use futures::future::{self, FutureExt};
use std::env;
use std::io;
use std::process::{ExitStatus, Stdio};
fn cat() -> Command {
let mut me = env::current_exe().unwrap();
me.pop();
if me.ends_with("deps") {
me.pop();
}
me.push("test-cat");
let mut cmd = Command::new(me);
cmd.stdin(Stdio::piped()).stdout(Stdio::piped());
cmd
}
async fn feed_cat(mut cat: Child, n: usize) -> io::Result<ExitStatus> {
let mut stdin = cat.stdin().take().unwrap();
let stdout = cat.stdout().take().unwrap();
// Produce n lines on the child's stdout.
let write = async {
for i in 0..n {
let bytes = format!("line {}\n", i).into_bytes();
stdin.write_all(&bytes).await.unwrap();
}
drop(stdin);
};
let read = async {
let mut reader = BufReader::new(stdout).lines();
let mut num_lines = 0;
// Try to read `n + 1` lines, ensuring the last one is empty
// (i.e. EOF is reached after `n` lines.
loop {
let data = reader
.next_line()
.await
.unwrap_or_else(|_| Some(String::new()))
.expect("failed to read line");
let num_read = data.len();
let done = num_lines >= n;
match (done, num_read) {
(false, 0) => panic!("broken pipe"),
(true, n) if n != 0 => panic!("extraneous data"),
_ => {
let expected = format!("line {}", num_lines);
assert_eq!(expected, data);
}
};
num_lines += 1;
if num_lines >= n {
break;
}
}
};
// Compose reading and writing concurrently.
future::join3(write, read, cat)
.map(|(_, _, status)| status)
.await
}
/// Check for the following properties when feeding stdin and
/// consuming stdout of a cat-like process:
///
/// - A number of lines that amounts to a number of bytes exceeding a
/// typical OS buffer size can be fed to the child without
/// deadlock. This tests that we also consume the stdout
/// concurrently; otherwise this would deadlock.
///
/// - We read the same lines from the child that we fed it.
///
/// - The child does produce EOF on stdout after the last line.
#[tokio::test]
async fn feed_a_lot() {
let child = cat().spawn().unwrap();
let status = feed_cat(child, 10000).await.unwrap();
assert_eq!(status.code(), Some(0));
}
#[tokio::test]
async fn wait_with_output_captures() {
let mut child = cat().spawn().unwrap();
let mut stdin = child.stdin().take().unwrap();
let write_bytes = b"1234";
let future = async {
stdin.write_all(write_bytes).await?;
drop(stdin);
let out = child.wait_with_output();
out.await
};
let output = future.await.unwrap();
assert!(output.status.success());
assert_eq!(output.stdout, write_bytes);
assert_eq!(output.stderr.len(), 0);
}
#[tokio::test]
async fn status_closes_any_pipes() {
// Cat will open a pipe between the parent and child.
// If `status_async` doesn't ensure the handles are closed,
// we would end up blocking forever (and time out).
let child = cat().status();
assert_ok!(child.await);
}
-63
View File
@@ -1,63 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::net::TcpStream;
use std::thread;
use std::io::{Read, Write, BufReader, BufWriter};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::copy;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
const N: usize = 1024;
drop(env_logger::try_init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let t2 = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let mut b = vec![0; msg.len() * N];
t!(s.read_exact(&mut b));
b
});
let mut expected = Vec::<u8>::new();
for _i in 0..N {
expected.extend(msg.as_bytes());
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
}
(expected, t2)
});
let clients = srv.incoming().take(2).collect();
let copied = clients.and_then(|clients| {
let mut clients = clients.into_iter();
let a = BufReader::new(clients.next().unwrap());
let b = BufWriter::new(clients.next().unwrap());
copy(a, b)
});
let (amt, _, _) = t!(copied.wait());
let (expected, t2) = t.join().unwrap();
let actual = t2.join().unwrap();
assert!(expected == actual);
assert_eq!(amt, msg.len() as u64 * 1024);
}
-69
View File
@@ -1,69 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_timer;
extern crate env_logger;
use tokio::prelude::*;
use tokio::runtime::{self, current_thread};
use tokio::timer::*;
use tokio_timer::clock::Clock;
use std::sync::mpsc;
use std::time::{Duration, Instant};
struct MockNow(Instant);
impl tokio_timer::clock::Now for MockNow {
fn now(&self) -> Instant {
self.0
}
}
#[test]
fn clock_and_timer_concurrent() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = runtime::Builder::new()
.clock(clock)
.build()
.unwrap();
let (tx, rx) = mpsc::channel();
rt.spawn({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn clock_and_timer_single_threaded() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = current_thread::Builder::new()
.clock(clock)
.build()
.unwrap();
rt.block_on({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
Ok(())
})
}).unwrap();
}
-42
View File
@@ -1,42 +0,0 @@
extern crate tokio;
extern crate futures;
use std::thread;
use std::net;
use futures::future;
use futures::prelude::*;
use futures::sync::oneshot;
use tokio::net::TcpListener;
use tokio::reactor::Reactor;
#[test]
fn tcp_doesnt_block() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
drop(core);
assert!(listener.incoming().wait().next().unwrap().is_err());
}
#[test]
fn drop_wakes() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
let (tx, rx) = oneshot::channel::<()>();
let t = thread::spawn(move || {
let incoming = listener.incoming();
let new_socket = incoming.into_future().map_err(|_| ());
let drop_tx = future::lazy(|| {
drop(tx);
future::ok(())
});
assert!(new_socket.join(drop_tx).wait().is_err());
});
drop(rx.wait());
drop(core);
t.join().unwrap();
}
-27
View File
@@ -1,27 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_executor;
extern crate tokio_timer;
use futures::sync::mpsc;
use tokio::util::StreamExt;
#[test]
fn enumerate() {
use futures::*;
let (mut tx, rx) = mpsc::channel(1);
std::thread::spawn(|| {
for i in 0..5 {
tx = tx.send(i * 2).wait().unwrap();
}
});
let result = rx.enumerate().collect();
assert_eq!(
result.wait(),
Ok(vec![(0, 0), (1, 2), (2, 4), (3, 6), (4, 8)])
);
}
-136
View File
@@ -1,136 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn hammer_old() {
let _ = env_logger::try_init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().into_future()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(mine.join(theirs).wait());
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
}).collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
}
struct Rd(Arc<TcpStream>);
struct Wr(Arc<TcpStream>);
impl io::Read for Rd {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
<&TcpStream>::read(&mut &*self.0, dst)
}
}
impl tokio_io::AsyncRead for Rd {
}
impl io::Write for Wr {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
<&TcpStream>::write(&mut &*self.0, src)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl tokio_io::AsyncWrite for Wr {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
#[test]
fn hammer_split() {
use tokio_io::io;
const N: usize = 100;
const ITER: usize = 10;
let _ = env_logger::try_init();
for _ in 0..ITER {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let cnt = Arc::new(AtomicUsize::new(0));
let mut rt = Runtime::new().unwrap();
fn split(socket: TcpStream, cnt: Arc<AtomicUsize>) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let cnt2 = cnt.clone();
let rd = io::read(rd, vec![0; 1])
.map(move |_| {
cnt2.fetch_add(1, Relaxed);
})
.map_err(|e| panic!("read error = {:?}", e));
let wr = io::write_all(wr, b"1")
.map(move |_| {
cnt.fetch_add(1, Relaxed);
})
.map_err(move |e| panic!("write error = {:?}", e));
tokio::spawn(rd);
tokio::spawn(wr);
}
rt.spawn({
let cnt = cnt.clone();
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(move |socket| {
split(socket, cnt.clone());
Ok(())
})
});
for _ in 0..N {
rt.spawn({
let cnt = cnt.clone();
TcpStream::connect(&addr)
.map_err(move |e| panic!("connect error = {:?}", e))
.map(move |socket| split(socket, cnt))
});
}
rt.shutdown_on_idle().wait().unwrap();
assert_eq!(N * 4, cnt.load(Relaxed));
}
}
-580
View File
@@ -1,580 +0,0 @@
extern crate tokio;
extern crate futures;
extern crate bytes;
use tokio::io::{AsyncRead, AsyncWrite};
use tokio::codec::*;
use bytes::{Bytes, BytesMut, BufMut};
use futures::{Stream, Sink, Poll};
use futures::Async::*;
use std::io;
use std::collections::VecDeque;
macro_rules! mock {
($($x:expr,)*) => {{
let mut v = VecDeque::new();
v.extend(vec![$($x),*]);
Mock { calls: v }
}};
}
#[test]
fn read_empty_io_yields_nothing() {
let mut io = FramedRead::new(mock!(), LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet_little_endian() {
let mut io = length_delimited::Builder::new()
.little_endian()
.new_read(mock! {
Ok(b"\x09\x00\x00\x00abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet_native_endian() {
let data = if cfg!(target_endian = "big") {
b"\x00\x00\x00\x09abcdefghi"
} else {
b"\x09\x00\x00\x00abcdefghi"
};
let mut io = length_delimited::Builder::new()
.native_endian()
.new_read(mock! {
Ok(data[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"\x00\x00\x00\x09abcdefghi");
data.extend_from_slice(b"\x00\x00\x00\x03123");
data.extend_from_slice(b"\x00\x00\x00\x0bhello world");
let mut io = FramedRead::new(mock! {
Ok(data.into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_multi_packet() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Ok(b"\x00\x09abc"[..].into()),
Ok(b"defghi"[..].into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_multi_frame_multi_packet() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Ok(b"\x00\x09abc"[..].into()),
Ok(b"defghi"[..].into()),
Ok(b"\x00\x00\x00\x0312"[..].into()),
Ok(b"3\x00\x00\x00\x0bhello world"[..].into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_multi_packet_wait() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09abc"[..].into()),
Err(would_block()),
Ok(b"defghi"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_multi_frame_multi_packet_wait() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09abc"[..].into()),
Err(would_block()),
Ok(b"defghi"[..].into()),
Err(would_block()),
Ok(b"\x00\x00\x00\x0312"[..].into()),
Err(would_block()),
Ok(b"3\x00\x00\x00\x0bhello world"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_incomplete_head() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
}, LengthDelimitedCodec::new());
assert!(io.poll().is_err());
}
#[test]
fn read_incomplete_head_multi() {
let mut io = FramedRead::new(mock! {
Err(would_block()),
Ok(b"\x00"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert!(io.poll().is_err());
}
#[test]
fn read_incomplete_payload() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00\x00\x09ab"[..].into()),
Err(would_block()),
Ok(b"cd"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert!(io.poll().is_err());
}
#[test]
fn read_max_frame_len() {
let mut io = length_delimited::Builder::new()
.max_frame_length(5)
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_update_max_frame_len_at_rest() {
let mut io = length_delimited::Builder::new()
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
io.decoder_mut().set_max_frame_length(5);
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_update_max_frame_len_in_flight() {
let mut io = length_delimited::Builder::new()
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcd"[..].into()),
Err(would_block()),
Ok(b"efghi"[..].into()),
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), NotReady);
io.decoder_mut().set_max_frame_length(5);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_one_byte_length_field() {
let mut io = length_delimited::Builder::new()
.length_field_length(1)
.new_read(mock! {
Ok(b"\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_header_offset() {
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_field_offset(4)
.new_read(mock! {
Ok(b"zzzz\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet_skip_none_adjusted() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"xx\x00\x09abcdefghi");
data.extend_from_slice(b"yy\x00\x03123");
data.extend_from_slice(b"zz\x00\x0bhello world");
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_field_offset(2)
.num_skip(0)
.length_adjustment(4)
.new_read(mock! {
Ok(data.into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"xx\x00\x09abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"yy\x00\x03123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"zz\x00\x0bhello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet_length_includes_head() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"\x00\x0babcdefghi");
data.extend_from_slice(b"\x00\x05123");
data.extend_from_slice(b"\x00\x0dhello world");
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_adjustment(-2)
.new_read(mock! {
Ok(data.into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn write_single_frame_length_adjusted() {
let mut io = length_delimited::Builder::new()
.length_adjustment(-2)
.new_write(mock! {
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_nothing_yields_nothing() {
let mut io = FramedWrite::new(
mock!(),
LengthDelimitedCodec::new()
);
assert!(io.poll_complete().unwrap().is_ready());
}
#[test]
fn write_single_frame_one_packet() {
let mut io = FramedWrite::new(mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_multi_frame_one_packet() {
let mut io = FramedWrite::new(mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(b"\x00\x00\x00\x03"[..].into()),
Ok(b"123"[..].into()),
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"hello world"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.start_send(Bytes::from("123")).unwrap().is_ready());
assert!(io.start_send(Bytes::from("hello world")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_multi_frame_multi_packet() {
let mut io = FramedWrite::new(mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
Ok(b"\x00\x00\x00\x03"[..].into()),
Ok(b"123"[..].into()),
Ok(Flush),
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"hello world"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.start_send(Bytes::from("123")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.start_send(Bytes::from("hello world")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_would_block() {
let mut io = FramedWrite::new(mock! {
Err(would_block()),
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_little_endian() {
let mut io = length_delimited::Builder::new()
.little_endian()
.new_write(mock! {
Ok(b"\x09\x00\x00\x00"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_with_short_length_field() {
let mut io = length_delimited::Builder::new()
.length_field_length(1)
.new_write(mock! {
Ok(b"\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_max_frame_len() {
let mut io = length_delimited::Builder::new()
.max_frame_length(5)
.new_write(mock! { });
assert_eq!(io.start_send(Bytes::from("abcdef")).unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_update_max_frame_len_at_rest() {
let mut io = length_delimited::Builder::new()
.new_write(mock! {
Ok(b"\x00\x00\x00\x06"[..].into()),
Ok(b"abcdef"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
io.encoder_mut().set_max_frame_length(5);
assert_eq!(io.start_send(Bytes::from("abcdef")).unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_update_max_frame_len_in_flight() {
let mut io = length_delimited::Builder::new()
.new_write(mock! {
Ok(b"\x00\x00\x00\x06"[..].into()),
Ok(b"ab"[..].into()),
Err(would_block()),
Ok(b"cdef"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
io.encoder_mut().set_max_frame_length(5);
assert!(io.poll_complete().unwrap().is_ready());
assert_eq!(io.start_send(Bytes::from("abcdef")).unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_zero() {
let mut io = length_delimited::Builder::new()
.new_write(mock! { });
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert_eq!(io.poll_complete().unwrap_err().kind(), io::ErrorKind::WriteZero);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn encode_overflow() {
// Test reproducing tokio-rs/tokio#681.
let mut codec = length_delimited::Builder::new().new_codec();
let mut buf = BytesMut::with_capacity(1024);
// Put some data into the buffer without resizing it to hold more.
let some_as = std::iter::repeat(b'a')
.take(1024)
.collect::<Vec<_>>();
buf.put_slice(&some_as[..]);
// Trying to encode the length header should resize the buffer if it won't fit.
codec.encode(Bytes::from("hello"), &mut buf).unwrap();
}
// ===== Test utils =====
fn would_block() -> io::Error {
io::Error::new(io::ErrorKind::WouldBlock, "would block")
}
struct Mock {
calls: VecDeque<io::Result<Op>>,
}
enum Op {
Data(Vec<u8>),
Flush,
}
use self::Op::*;
impl io::Read for Mock {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(Op::Data(data))) => {
debug_assert!(dst.len() >= data.len());
dst[..data.len()].copy_from_slice(&data[..]);
Ok(data.len())
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
}
impl AsyncRead for Mock {
}
impl io::Write for Mock {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(Op::Data(data))) => {
let len = data.len();
assert!(src.len() >= len, "expect={:?}; actual={:?}", data, src);
assert_eq!(&data[..], &src[..len]);
Ok(len)
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
fn flush(&mut self) -> io::Result<()> {
match self.calls.pop_front() {
Some(Ok(Op::Flush)) => {
Ok(())
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(()),
}
}
}
impl AsyncWrite for Mock {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(Ready(()))
}
}
impl<'a> From<&'a [u8]> for Op {
fn from(src: &'a [u8]) -> Op {
Op::Data(src.into())
}
}
impl From<Vec<u8>> for Op {
fn from(src: Vec<u8>) -> Op {
Op::Data(src)
}
}
-88
View File
@@ -1,88 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate tokio_threadpool;
extern crate bytes;
use std::io;
use std::net::Shutdown;
use bytes::{BytesMut, BufMut};
use futures::{Future, Stream, Sink};
use tokio::net::{TcpListener, TcpStream};
use tokio_codec::{Encoder, Decoder};
use tokio_io::io::{write_all, read};
use tokio_threadpool::Builder;
pub struct LineCodec;
impl Decoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<BytesMut>, io::Error> {
match buf.iter().position(|&b| b == b'\n') {
Some(i) => Ok(Some(buf.split_to(i + 1).into())),
None => Ok(None),
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() == 0 {
Ok(None)
} else {
let amt = buf.len();
Ok(Some(buf.split_to(amt)))
}
}
}
impl Encoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn encode(&mut self, item: BytesMut, into: &mut BytesMut) -> io::Result<()> {
into.put(&item[..]);
Ok(())
}
}
#[test]
fn echo() {
drop(env_logger::try_init());
let pool = Builder::new()
.pool_size(1)
.build();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
let addr = listener.local_addr().unwrap();
let sender = pool.sender().clone();
let srv = listener.incoming().for_each(move |socket| {
let (sink, stream) = LineCodec.framed(socket).split();
sender.spawn(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
Ok(())
});
pool.sender().spawn(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
let client = TcpStream::connect(&addr);
let client = client.wait().unwrap();
let (client, _) = write_all(client, b"a\n").wait().unwrap();
let (client, buf, amt) = read(client, vec![0; 1024]).wait().unwrap();
assert_eq!(amt, 2);
assert_eq!(&buf[..2], b"a\n");
let (client, _) = write_all(client, b"\n").wait().unwrap();
let (client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"\n");
let (client, _) = write_all(client, b"b").wait().unwrap();
client.shutdown(Shutdown::Write).unwrap();
let (_client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"b");
}
-88
View File
@@ -1,88 +0,0 @@
#![cfg(unix)]
extern crate env_logger;
extern crate futures;
extern crate libc;
extern crate mio;
extern crate tokio;
extern crate tokio_io;
use std::fs::File;
use std::io::{self, Write};
use std::os::unix::io::{AsRawFd, FromRawFd};
use std::thread;
use std::time::Duration;
use mio::event::Evented;
use mio::unix::{UnixReady, EventedFd};
use mio::{PollOpt, Ready, Token};
use tokio::reactor::{Handle, PollEvented2};
use tokio_io::io::read_to_end;
use futures::Future;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
struct MyFile(File);
impl MyFile {
fn new(file: File) -> MyFile {
unsafe {
let r = libc::fcntl(file.as_raw_fd(), libc::F_SETFL, libc::O_NONBLOCK);
assert!(r != -1, "fcntl error: {}", io::Error::last_os_error());
}
MyFile(file)
}
}
impl io::Read for MyFile {
fn read(&mut self, bytes: &mut [u8]) -> io::Result<usize> {
self.0.read(bytes)
}
}
impl Evented for MyFile {
fn register(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).register(poll, token, interest | hup, opts)
}
fn reregister(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).reregister(poll, token, interest | hup, opts)
}
fn deregister(&self, poll: &mio::Poll) -> io::Result<()> {
EventedFd(&self.0.as_raw_fd()).deregister(poll)
}
}
#[test]
fn hup() {
drop(env_logger::try_init());
let handle = Handle::default();
unsafe {
let mut pipes = [0; 2];
assert!(libc::pipe(pipes.as_mut_ptr()) != -1,
"pipe error: {}", io::Error::last_os_error());
let read = File::from_raw_fd(pipes[0]);
let mut write = File::from_raw_fd(pipes[1]);
let t = thread::spawn(move || {
write.write_all(b"Hello!\n").unwrap();
write.write_all(b"Good bye!\n").unwrap();
thread::sleep(Duration::from_millis(100));
});
let source = PollEvented2::new_with_handle(MyFile::new(read), &handle).unwrap();
let reader = read_to_end(source, Vec::new());
let (_, content) = t!(reader.wait());
assert_eq!(&b"Hello!\nGood bye!\n"[..], &content[..]);
t.join().unwrap();
}
}
-89
View File
@@ -1,89 +0,0 @@
extern crate futures;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_tcp;
use tokio_reactor::Reactor;
use tokio_tcp::TcpListener;
use futures::{Future, Stream};
use futures::executor::{spawn, Notify, Spawn};
use std::mem;
use std::net::TcpStream;
use std::sync::{Arc, Mutex};
#[test]
fn test_drop_on_notify() {
// When the reactor receives a kernel notification, it notifies the
// task that holds the associated socket. If this notification results in
// the task being dropped, the socket will also be dropped.
//
// Previously, there was a deadlock scenario where the reactor, while
// notifying, held a lock and the task being dropped attempted to acquire
// that same lock in order to clean up state.
//
// To simulate this case, we create a fake executor that does nothing when
// the task is notified. This simulates an executor in the process of
// shutting down. Then, when the task handle is dropped, the task itself is
// dropped.
struct MyNotify;
type Task = Mutex<Spawn<Box<Future<Item = (), Error = ()>>>>;
impl Notify for MyNotify {
fn notify(&self, _: usize) {
// Do nothing
}
fn clone_id(&self, id: usize) -> usize {
let ptr = id as *const Task;
let task = unsafe { Arc::from_raw(ptr) };
mem::forget(task.clone());
mem::forget(task);
id
}
fn drop_id(&self, id: usize) {
let ptr = id as *const Task;
let _ = unsafe { Arc::from_raw(ptr) };
}
}
let addr = "127.0.0.1:0".parse().unwrap();
let mut reactor = Reactor::new().unwrap();
// Create a listener
let listener = TcpListener::bind(&addr).unwrap();
let addr = listener.local_addr().unwrap();
// Define a task that just drains the listener
let task = Box::new({
listener.incoming()
.for_each(|_| Ok(()))
.map_err(|_| panic!())
}) as Box<Future<Item = (), Error = ()>>;
let task = Arc::new(Mutex::new(spawn(task)));
let notify = Arc::new(MyNotify);
let mut enter = tokio_executor::enter().unwrap();
tokio_reactor::with_default(&reactor.handle(), &mut enter, |_| {
let id = &*task as *const Task as usize;
task.lock().unwrap()
.poll_future_notify(&notify, id)
.unwrap();
});
drop(task);
// Establish a connection to the acceptor
let _s = TcpStream::connect(&addr).unwrap();
reactor.turn(None).unwrap();
}
-516
View File
@@ -1,516 +0,0 @@
extern crate tokio;
extern crate env_logger;
extern crate futures;
use futures::sync::oneshot;
use std::sync::{Arc, Mutex, atomic};
use std::thread;
use tokio::io;
use tokio::net::{TcpStream, TcpListener};
use tokio::prelude::future::lazy;
use tokio::prelude::*;
use tokio::runtime::Runtime;
// this import is used in all child modules that have it in scope
// from importing super::*, but the compiler doesn't realise that
// and warns about it.
pub use futures::future::Executor;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
fn create_client_server_future() -> Box<Future<Item=(), Error=()> + Send> {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| panic!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| ())
.map_err(|e| panic!("write err = {:?}", e))
})
})
.map(|_| ());
let client = client
.map_err(|e| panic!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| ())
.map_err(|e| panic!("read err = {:?}", e))
});
let future = server.join(client)
.map(|_| ());
Box::new(future)
}
#[test]
fn runtime_tokio_run() {
let _ = env_logger::try_init();
tokio::run(create_client_server_future());
}
#[test]
fn runtime_single_threaded() {
let _ = env_logger::try_init();
let mut runtime = tokio::runtime::current_thread::Runtime::new()
.unwrap();
runtime.block_on(create_client_server_future()).unwrap();
runtime.run().unwrap();
}
#[test]
fn runtime_single_threaded_block_on() {
let _ = env_logger::try_init();
tokio::runtime::current_thread::block_on_all(create_client_server_future()).unwrap();
}
mod runtime_single_threaded_block_on_all {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>),
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let msg = tokio::runtime::current_thread::block_on_all(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn spawn() {
test(|f| { tokio::spawn(f); })
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
}
mod runtime_single_threaded_racy {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(
tokio::runtime::current_thread::Handle,
Box<Future<Item=(), Error=()> + Send>,
),
{
let (trigger, exit) = futures::sync::oneshot::channel();
let (handle_tx, handle_rx) = ::std::sync::mpsc::channel();
let jh = ::std::thread::spawn(move || {
let mut rt = tokio::runtime::current_thread::Runtime::new().unwrap();
handle_tx.send(rt.handle()).unwrap();
// don't exit until we are told to
rt.block_on(exit.map_err(|_| ())).unwrap();
// run until all spawned futures (incl. the "exit" signal future) have completed.
rt.run().unwrap();
});
let (tx, rx) = futures::sync::oneshot::channel();
let handle = handle_rx.recv().unwrap();
spawn(handle, Box::new(futures::future::lazy(move || {
tx.send(()).unwrap();
Ok(())
})));
// signal runtime thread to exit
trigger.send(()).unwrap();
// wait for runtime thread to exit
jh.join().unwrap();
assert_eq!(rx.wait().unwrap(), ());
}
#[test]
fn spawn() {
test(|handle, f| { handle.spawn(f).unwrap(); })
}
#[test]
fn execute() {
test(|handle, f| { handle.execute(f).unwrap(); })
}
}
mod runtime_multi_threaded {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(&mut Runtime) + Send + 'static,
{
let _ = env_logger::try_init();
let mut runtime = tokio::runtime::Builder::new()
.build()
.unwrap();
spawn(&mut runtime);
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn() {
test(|rt| { rt.spawn(create_client_server_future()); });
}
#[test]
fn execute() {
test(|rt| { rt.executor().execute(create_client_server_future()).unwrap(); });
}
}
#[test]
fn block_on_timer() {
use std::time::{Duration, Instant};
use tokio::timer::{Delay, Error};
fn after_1s<T>(x: T) -> Box<Future<Item = T, Error = Error> + Send>
where
T: Send + 'static,
{
Box::new(Delay::new(Instant::now() + Duration::from_millis(100)).map(move |_| x))
}
let mut runtime = Runtime::new().unwrap();
assert_eq!(runtime.block_on(after_1s(42)).unwrap(), 42);
runtime.shutdown_on_idle().wait().unwrap();
}
mod from_block_on {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>) + Send + 'static,
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
let msg = runtime
.block_on(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
#[test]
fn spawn() {
test(|f| {
tokio::spawn(f);
})
}
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
use std::time::Duration;
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
runtime
.block_on(rx.then(move |_| {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<_, ()>(())
}))
.unwrap();
assert_eq!(1, *cnt.lock().unwrap());
runtime.shutdown_on_idle().wait().unwrap();
}
mod many {
use super::*;
const ITER: usize = 200;
fn test<F>(spawn: F)
where
F: Fn(&mut Runtime, Box<Future<Item=(), Error=()> + Send>),
{
let cnt = Arc::new(Mutex::new(0));
let mut runtime = Runtime::new().unwrap();
for _ in 0..ITER {
let c = cnt.clone();
spawn(&mut runtime, Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
}
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(ITER, *cnt.lock().unwrap());
}
#[test]
fn spawn() {
test(|rt, f| { rt.spawn(f); })
}
#[test]
fn execute() {
test(|rt, f| {
rt.executor()
.execute(f)
.unwrap();
})
}
}
mod from_block_on_all {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>) + Send + 'static,
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let runtime = Runtime::new().unwrap();
let msg = runtime
.block_on_all(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
#[test]
fn spawn() {
test(|f| { tokio::spawn(f); })
}
}
mod nested_enter {
use super::*;
use tokio::runtime::current_thread;
use std::panic;
fn test<F1, F2>(first: F1, nested: F2)
where
F1: Fn(Box<Future<Item=(), Error=()> + Send>) + Send + 'static,
F2: Fn(Box<Future<Item=(), Error=()> + Send>) + panic::UnwindSafe + Send + 'static,
{
let panicked = Arc::new(Mutex::new(false));
let panicked2 = panicked.clone();
// Since this is testing panics in other threads, printing about panics
// is noisy and can give the impression that the test is ignoring panics.
//
// It *is* ignoring them, but on purpose.
let prev_hook = panic::take_hook();
panic::set_hook(Box::new(|info| {
let s = info.to_string();
if s.starts_with("panicked at 'nested ")
|| s.starts_with("panicked at 'Multiple executors at once")
{
// expected, noop
} else {
println!("{}", s);
}
}));
first(Box::new(lazy(move || {
panic::catch_unwind(move || {
nested(Box::new(lazy(|| { Ok::<(), ()>(()) })))
}).expect_err("nested should panic");
*panicked2.lock().unwrap() = true;
Ok::<(), ()>(())
})));
panic::set_hook(prev_hook);
assert!(*panicked.lock().unwrap(), "nested call should have panicked");
}
fn threadpool_new() -> Runtime {
Runtime::new().expect("rt new")
}
#[test]
fn run_in_run() {
test(tokio::run, tokio::run);
}
#[test]
fn threadpool_block_on_in_run() {
test(tokio::run, |fut| {
let mut rt = threadpool_new();
rt.block_on(fut).unwrap();
});
}
#[test]
fn threadpool_block_on_all_in_run() {
test(tokio::run, |fut| {
let rt = threadpool_new();
rt.block_on_all(fut).unwrap();
});
}
#[test]
fn current_thread_block_on_all_in_run() {
test(tokio::run, |fut| {
current_thread::block_on_all(fut).unwrap();
});
}
}
#[test]
fn runtime_reactor_handle() {
#![allow(deprecated)]
use futures::Stream;
use std::net::{
TcpListener as StdListener,
TcpStream as StdStream,
};
let rt = Runtime::new().unwrap();
let std_listener = StdListener::bind("127.0.0.1:0").unwrap();
let tk_listener = TcpListener::from_std(std_listener, rt.handle()).unwrap();
let addr = tk_listener.local_addr().unwrap();
// Spawn a thread since we are avoiding the runtime
let th = thread::spawn(|| {
for _ in tk_listener.incoming().take(1).wait() {
}
});
let _ = StdStream::connect(&addr).unwrap();
th.join().unwrap();
}
#[test]
fn after_start_and_before_stop_is_called() {
let _ = env_logger::try_init();
let after_start = Arc::new(atomic::AtomicUsize::new(0));
let before_stop = Arc::new(atomic::AtomicUsize::new(0));
let after_inner = after_start.clone();
let before_inner = before_stop.clone();
let runtime = tokio::runtime::Builder::new()
.after_start(move || { after_inner.clone().fetch_add(1, atomic::Ordering::Relaxed); })
.before_stop(move || { before_inner.clone().fetch_add(1, atomic::Ordering::Relaxed); })
.build()
.unwrap();
runtime.block_on_all(create_client_server_future()).unwrap();
assert!(after_start.load(atomic::Ordering::Relaxed) > 0);
assert!(before_stop.load(atomic::Ordering::Relaxed) > 0);
}
-116
View File
@@ -1,116 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use tokio::prelude::*;
use tokio::timer::*;
use std::sync::mpsc;
use std::time::{Duration, Instant};
#[test]
fn timer_with_runtime() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(100);
let (tx, rx) = mpsc::channel();
tokio::run({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn starving() {
use futures::{task, Poll, Async};
let _ = env_logger::try_init();
struct Starve(Delay, u64);
impl Future for Starve {
type Item = u64;
type Error = ();
fn poll(&mut self) -> Poll<Self::Item, ()> {
if self.0.poll().unwrap().is_ready() {
return Ok(self.1.into());
}
self.1 += 1;
task::current().notify();
Ok(Async::NotReady)
}
}
let when = Instant::now() + Duration::from_millis(20);
let starve = Starve(Delay::new(when), 0);
let (tx, rx) = mpsc::channel();
tokio::run({
starve
.and_then(move |_ticks| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn deadline() {
use futures::future;
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(20);
let (tx, rx) = mpsc::channel();
#[allow(deprecated)]
tokio::run({
future::empty::<(), ()>()
.deadline(when)
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn timeout() {
use futures::future;
let _ = env_logger::try_init();
let (tx, rx) = mpsc::channel();
tokio::run({
future::empty::<(), ()>()
.timeout(Duration::from_millis(20))
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
-29
View File
@@ -1,29 +0,0 @@
[package]
name = "tokio-async-await"
# When releasing to crates.io:
# - Update html_root_url.
version = "0.1.5"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-async-await/0.1.3"
description = """
Experimental async/await support for Tokio
"""
categories = ["asynchronous"]
[features]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = ["futures/nightly"]
[dependencies]
futures = "0.1.23"
tokio-io = { version = "0.1.7", path = "../tokio-io" }
[dev-dependencies]
bytes = "0.4.9"
tokio = { version = "0.1.8", path = ".." }
hyper = "0.12.8"
-52
View File
@@ -1,52 +0,0 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
Copyright (c) 2016 futures-rs authors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
-55
View File
@@ -1,55 +0,0 @@
# Tokio async/await preview
This crate provides a preview of Tokio with async / await support. It is a shim
layer on top of `tokio`.
**This crate requires Rust nightly and does not provide API stability
guarantees. You are living on the edge here.**
## Usage
To use this crate, you need to start with a Rust 2018 edition crate, with rustc
1.33.0-nightly or later.
Add this to your `Cargo.toml`:
```toml
# In the `[packages]` section
edition = "2018"
# In the `[dependencies]` section
tokio = {version = "0.1.0", features = ["async-await-preview"]}
```
Then, get started. In your application, add:
```rust
// The nightly features that are commonly needed with async / await
#![feature(await_macro, async_await, futures_api)]
// This pulls in the `tokio-async-await` crate. While Rust 2018 doesn't require
// `extern crate`, we need to pull in the macros.
#[macro_use]
extern crate tokio;
fn main() {
// And we are async...
tokio::run_async(async {
println!("Hello");
});
}
```
Because nightly is required, run the app with `cargo +nightly run`
Check the [examples](examples) directory for more.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-2
View File
@@ -1,2 +0,0 @@
[build]
target-dir = "../../target"
-49
View File
@@ -1,49 +0,0 @@
[package]
name = "examples"
edition = "2018"
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
# Break out of the parent workspace
[workspace]
[[bin]]
name = "chat"
path = "src/chat.rs"
[[bin]]
name = "echo_client"
path = "src/echo_client.rs"
[[bin]]
name = "echo_server"
path = "src/echo_server.rs"
[[bin]]
name = "hyper"
path = "src/hyper.rs"
[dependencies]
tokio = { version = "0.1.0", path = "../..", features = ["async-await-preview"] }
futures = "0.1.23"
bytes = "0.4.9"
hyper = "0.12.8"
# Avoid using crates.io for Tokio dependencies
[patch.crates-io]
tokio = { path = "../.." }
tokio-async-await = { path = "../" }
tokio-codec = { path = "../../tokio-codec" }
tokio-current-thread = { path = "../../tokio-current-thread" }
tokio-executor = { path = "../../tokio-executor" }
tokio-fs = { path = "../../tokio-fs" }
tokio-io = { path = "../../tokio-io" }
tokio-reactor = { path = "../../tokio-reactor" }
tokio-signal = { path = "../../tokio-signal" }
tokio-tcp = { path = "../../tokio-tcp" }
tokio-threadpool = { path = "../../tokio-threadpool" }
tokio-timer = { path = "../../tokio-timer" }
tokio-tls = { path = "../../tokio-tls" }
tokio-udp = { path = "../../tokio-udp" }
tokio-uds = { path = "../../tokio-uds" }
-5
View File
@@ -1,5 +0,0 @@
# Tokio async/await examples
These are a separate crate in order to work around some cargo bugs. It also
allows `[patch]` to be used in `Cargo.toml` to ensure the correct lib versions
are being pulled in.
-135
View File
@@ -1,135 +0,0 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate futures; // v0.1
use tokio::codec::{LinesCodec, Decoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use std::collections::HashMap;
use std::io;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<String>;
struct Shared {
peers: HashMap<SocketAddr, Tx>,
}
impl Shared {
/// Create a new, empty, instance of `Shared`.
fn new() -> Self {
Shared {
peers: HashMap::new(),
}
}
}
async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()> {
let addr = stream.peer_addr().unwrap();
let mut lines = LinesCodec::new().framed(stream);
// Extract the peer's name
let name = match await!(lines.next()) {
Some(name) => name?,
None => {
// Disconnected early
return Ok(());
}
};
println!("`{}` is joining the chat", name);
let (tx, mut rx) = mpsc::unbounded();
// Register the socket
state.lock().unwrap()
.peers.insert(addr, tx);
// Split the `lines` handle into send and recv handles. This allows spawning
// separate tasks.
let (mut lines_tx, mut lines_rx) = lines.split();
// Spawn a task that receives all lines broadcasted to us from other peers
// and writes it to the client.
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line));
}
});
// Use the current task to read lines from the socket and broadcast them to
// other peers.
while let Some(message) = await!(lines_rx.next()) {
// TODO: Error handling
let message = message.unwrap();
let mut line = name.clone();
line.push_str(": ");
line.push_str(&message);
line.push_str("\r\n");
let state = state.lock().unwrap();
for (peer_addr, tx) in &state.peers {
if *peer_addr != addr {
// TODO: Error handling
tx.unbounded_send(line.clone()).unwrap();
}
}
}
// Remove the client from the shared state. Doing so will also result in the
// tx task to terminate.
state.lock().unwrap()
.peers.remove(&addr)
.expect("bug");
Ok(())
}
fn main() {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse().unwrap();
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
println!("server running on localhost:6142");
// Start the Tokio runtime.
tokio::run_async(async move {
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = match stream {
Ok(stream) => stream,
Err(_) => continue,
};
let state = state.clone();
tokio::spawn_async(async move {
if let Err(_) = await!(process(stream, state)) {
eprintln!("failed to process connection");
}
});
}
});
}
@@ -1,53 +0,0 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
use tokio::net::TcpStream;
use tokio::prelude::*;
use std::io;
use std::net::SocketAddr;
const MESSAGES: &[&str] = &[
"hello",
"world",
"one two three",
];
async fn run_client(addr: &SocketAddr) -> io::Result<()> {
let mut stream = await!(TcpStream::connect(addr))?;
// Buffer to read into
let mut buf = [0; 128];
for msg in MESSAGES {
println!(" > write = {:?}", msg);
// Write the message to the server
await!(stream.write_all_async(msg.as_bytes()))?;
// Read the message back from the server
await!(stream.read_exact_async(&mut buf[..msg.len()]))?;
assert_eq!(&buf[..msg.len()], msg.as_bytes());
}
Ok(())
}
fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Connect to the echo serveer
tokio::run_async(async move {
match await!(run_client(&addr)) {
Ok(_) => println!("done."),
Err(e) => eprintln!("echo client failed; error = {:?}", e),
}
});
}

Some files were not shown because too many files have changed in this diff Show More