Compare commits

...
Author SHA1 Message Date
Jon Gjengset 5efe31f2ed Prepare for release of 0.2.0-alpha.6 (#1617)
Note that `tokio-timer` and `tokio-tls` become 0.3.0-alpha.6 (not 0.2.0)
2019-09-30 18:35:52 -04:00
Jon Gjengset 5ce5a0a0e0 Fix for rust-lang/rust#64477 (#1618)
`foo(format!(...)).await` no longer compiles. There's a fix in
rust-lang/rust#64856, but this works around the problem.
2019-09-30 17:17:14 -04:00
Jon Gjengset 5fd5329497 Create BufStream from a BufReader + BufWriter (#1609)
This is handy if developers want to construct the inner buffers with a
particular capacity, and still end up with a `BufStream` at the end.
2019-09-30 14:22:59 -04:00
Taiki Endo 3b8ee2d991 chore: update futures-preview to 0.3.0-alpha.19 (#1610) 2019-09-30 13:32:37 -04:00
Jon Gjengset 7c341f45e0 chore: move CI to beta (#1615) 2019-09-27 09:51:45 -07:00
Jon Gjengset 611b4e11a7 Make Barrier::wait future Send (#1611)
It wasn't before. Now it is. And that is better.
2019-09-26 18:26:24 -04:00
Taiki Endo 159abb375f chore: update pin-project to 0.4 (#1603) 2019-09-27 04:51:28 +09:00
Carl Lerche 032b39487c sync: add spin_loop_hint to atomic waker (#1608)
The algorithm backing `AtomicWaker` effectively uses a spin lock backed
by notifying & yielding the current task. This adds a `spin_lock_hint`
annotation to cover this case.

While, in practice, the omission of `spin_lock_hint` would not cause
problems, there are platforms that do not handle spin locks very well
and could enter a deadlock in pathological cases.
2019-09-26 15:16:34 -04:00
Hung-I Wang b71b7b36be fs: update the doc comment of File::sync_data (#1596) 2019-09-25 09:05:50 -07:00
Taiki Endo c4567f741a io: add get_*/into_inner methods to BufStream (#1598) 2019-09-25 09:17:43 -04:00
Sean McArthur 18cef1901f tokio: add rt-current-thread optional feature
- Adds a minimum `rt-current-thread` optional feature that exports
  `tokio::runtime::current_thread`.
- Adds a `macros` optional feature to enable the `#[tokio::main]` and
  `#[tokio::test]` attributes.
- Adjusts `#[tokio::main]` macro to select a runtime "automatically" if
  a specific strategy isn't specified. Allows using the macro with only
  the rt-current-thread feature.
2019-09-24 12:17:04 -07:00
Taiki Endo c81447fdcc io: remove unsafe pin-projections and remove manual Unpin implementations (#1588)
* Removes most pin-projection related unsafe code.

* Removes manual Unpin implementations.
  As references always implement Unpin, there is no need to implement
  Unpin manually.

* Adds tests to check that Unpin requirement does not change accidentally 
  because changing Unpin requirements will be breaking changes.
2019-09-25 01:17:06 +09:00
Taiki Endo d50d050fae net: fix build-tests for uds (#1589) 2019-09-24 02:47:39 +09:00
Taiki Endo 3a55aba251 macros: add build tests for #[tokio::main] and #[tokio::test] (#1591) 2019-09-23 04:09:30 +09:00
Taiki Endo ddbb0c3836 macros: fix handling of arguments of #[tokio::main] attribute (#1578) 2019-09-23 03:05:04 +09:00
Taiki Endo 376d63867a chore: update pin-project to 0.4.0-beta.1 (#1586) 2019-09-23 01:52:14 +09:00
Taiki Endo eb2d0fbcd1 net: use Box::pin instead of Pin::new(Box::new) (#1587) 2019-09-22 09:28:55 -07:00
Jonathan Bastien-Filiatrault 695165feac timer: 32 bit ARM only has 32 bit atomics. (#1581) 2019-09-20 13:25:49 -07:00
Kirill Mironov ff186a4d03 tokio: add process feature (#1561) 2019-09-19 19:03:58 -07:00
Jon Gjengset 6611b32cce Export sync::Barrier from tokio::sync (#1577) 2019-09-19 21:32:35 -04:00
Carl Lerche 80ba2a4ff6 Release 0.2.0 alpha.5 (#1576) 2019-09-19 13:39:35 -07:00
Carl Lerche 8d09f61d33 net: fix build with only process (#1575) 2019-09-19 12:38:15 -07:00
Carl Lerche 815173f8e5 chore: rm tokio-buf (#1574)
The crate has not been updated and it does not seem like it is a good
path forward.
2019-09-19 12:11:21 -07:00
Markus Westerlind 34e388619f timer: delay_for should use tokio_timer::clock::now (#1572) 2019-09-19 11:20:18 -07:00
Jon Gjengset 9d5af20bcf Enable buffering both reads and writes (#1558)
`BufWriter` and `BufReader` did not previously forward the "opposite" trait (`AsyncRead` for `BufWriter` and `AsyncWrite` for `BufReader`). This meant that there was no way to have both directions buffered at once. This patch fixes that, and introduces a convenience type + constructor for this double-wrapped construct.
2019-09-19 14:17:15 -04:00
Jon Gjengset 613fde2637 sync: add Barrier primitive (#1571)
This adds `Barrier` to `tokio-sync`, which is an asynchronous alternative to [`std::sync::Barrier`](https://doc.rust-lang.org/std/sync/struct.Barrier.html). It is a synchronization primitive that allows multiple futures to "rendezvous" at certain points in their execution.
2019-09-19 14:16:56 -04:00
Jonathan Bastien-Filiatrault 22a3b10171 executor: fix blocking pool bug re: thread shutdown (#1562)
Currently, when threads in the blocking pool shutdown due to being idle
the counter tracking threads is not decremented. This prevents new threads
from being spawned to replace the shutdown threads.
2019-09-19 11:12:04 -07:00
Jon Gjengset e3415d8d61 sync: Make Lock more similar to std::sync::Mutex (#1573)
This renames `Lock` to `Mutex`, and brings the API more in line with `std::sync::Mutex`.

In partcular, locking now only takes `&self`, with the expectation that you place the `Mutex` in an `Arc` (or something similar) to share it between threads.

Fixes #1544.
Part of #1210.
2019-09-19 11:46:52 -04:00
Taiki Endo d1f60ac4c6 chore: deny warnings for doc tests (#1539) 2019-09-19 15:50:12 +09:00
Taiki Endo e2161502ad chore: fix clippy check failure (#1569) 2019-09-18 10:19:44 -07:00
yjh ab785bfba7 Update README.md (#1545)
change url's `version` to `latest`.
2019-09-17 10:54:22 -04:00
Lucio Franco 5f2f3f076d Add broken feature to old benchmarks (#1555)
Signed-off-by: Lucio Franco <[email protected]>
2019-09-13 14:04:23 -04:00
Taiki Endo efb27731ad timer: use our own AtomicU64 on targets with target_has_atomic less than 64 (#1538) 2019-09-13 10:18:32 -07:00
cynecx 578a9aec16 sync: replace deprecated mem::uninitialized usage with MaybeUninit (#1540) 2019-09-13 10:03:03 -07:00
Jonathan Bastien-Filiatrault 5b8fc19701 fs: propagate flush for stdout / stderr. (#1528) 2019-09-13 09:58:18 -07:00
Geoff Shannon c0a64d67ca chore: fix docs links (#1523) 2019-09-13 09:46:19 -07:00
Carl Lerche 6369d0f4f2 chore: add stability note to readme. (#1554) 2019-09-13 09:12:30 -07:00
Kirill Mironov f69ee652e6 tls: fix new temporary lifetime rustc error [E0597] (#1547)
Fixes: #1546
Signed-off-by: Kirill Mironov <[email protected]>
2019-09-11 10:22:04 -07:00
Jon Gjengset 9b3f8564af tls: Add get_ref and get_mut (#1537) 2019-09-04 17:45:53 -04:00
Ivan Petkov 9766cd644f process: omit several future types in favor of async/await (#1526) 2019-08-31 13:02:27 -07:00
Carl Lerche 431d4857e8 io: add Send / Sync impls for ReadHalf / WriteHalf (#1525) 2019-08-31 12:18:55 -07:00
Fenhl 26432355d5 tokio-process: Implement From<StdCommand> for Command (#1513) 2019-08-31 11:45:41 -07:00
Carl Lerche 2f91c85ad8 io: bring back split utility (#1521)
Bring back `split` utility as a free fn instead of a method on
`AsyncRead`. This utility wraps the `stream` in an `Arc` and uses mutual
exclusion to ensure correct access.

Additionally, the specialized `split_mut` fn on TcpStream and UdsStream
is promoted to `split`.
2019-08-30 20:46:07 -07:00
Fenhl 951827229a Add platform-specific methods to Command (#1516) 2019-08-30 18:28:41 -07:00
Geoff Shannon 383bb0a143 test: fix assert format args (#1520) 2019-08-30 14:03:37 -07:00
Benjamin Saunders d2bd6f5002 timer: Rename sleep to delay_for, reexport from tokio (#1518) 2019-08-30 10:23:54 -07:00
Carl Lerche 6a94d2cf4f tls: bump to v0.3.0-alpha.4 (#1515) 2019-08-30 10:20:44 -07:00
kellerkindt 4f99470d46 chore: fix compile error on latest nightly (#1512) 2019-08-30 09:15:05 -07:00
Jarred Nicholls 3d9134d13e executor: shut down idle threads in the blocking pool (#1514) 2019-08-30 08:26:16 -07:00
Sean McArthur 15dc0563b7 prepare v0.2.0-alpha.4 (#1509) 2019-08-29 12:59:10 -07:00
Sean McArthur 4e26258ac3 Re-add temporarily TcpStream::connect_std (#1508) 2019-08-29 11:45:17 -07:00
Carl Lerche a59e096c47 prepare v0.2.0-alpha.3 release (#1505) 2019-08-28 15:04:42 -07:00
Carl Lerche fc1640891e net: perform DNS lookup on connect / bind. (#1499)
A sealed `net::ToSocketAddrs` trait is added. This trait is not intended
to be used by users. Instead, it is an argument to `connect` and `bind`
functions.

The operating system's DNS lookup functionality is used. Blocking
operations are performed on a thread pool in order to avoid blocking the
runtime.
2019-08-28 13:25:50 -07:00
Jakub Beránek de9f05d4d3 docs: fix wording in tokio_process::Child documentation (#1502)
Fixes: #1494
2019-08-28 11:56:42 -04:00
Eliza Weisman 9c31797a08 net: switch from log to tracing (#1455)
* net: switch from `log` to `tracing`.

Motivation:

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime.

Solution:

This branch replaces the use of `log` in `tokio-net` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation.

Notes:

I removed the timing in `Reactor::poll` in favor of simply adding a
`#[tracing::instrument]` attribute. Since the generated `tracing` span
will have enter and exit events, a `tracing::Subscriber`
implemementation can use those to record timestamps, and process that
timing data in a much more sophisticated manner than including it in a
log line.

We can add the timestamps back if they're desired.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-27 17:53:57 -07:00
Ömer Sinan Ağacan d1c58b7940 tokio: export RunError in tokio::runtime::current_thread (#1487)
This type is used in return type of `Runtime::run`, but because the type
was not exported it was opaque in the documentation of `Runtime`.
2019-08-27 12:26:48 -07:00
Jacob Pratt 5f74a99ea3 implement spawn_with_handle in tokio_executor (#1492)
This code directly relies on `future-preview`'s `RemoteHandle`, and
exposes it via a `spawn_with_handle` method that is identical to
`future-preview`'s implementation.

Related: #1180
2019-08-27 12:26:11 -07:00
Carl Lerche 08e20fcf6a fs: add support for non-threadpool executors (#1495)
Provides a thread pool dedicated to running blocking operations (#588)
and update `tokio-fs` to use this pool.

In an effort to make incremental progress, this is an initial step
towards a final solution. First, it provides a very basic pool
implementation with the intend that the pool will be
replaced before the final release. Second, it updates `tokio-fs` to
always use this blocking pool instead of conditionally using
`threadpool::blocking`. Issue #588 contains additional discussion around
potential improvements to the "blocking for all" strategy.

The implementation provided here builds on work started in #954 and
continued in #1045. The general idea is th same as #1045, but the PR
improves on some of the details:

* The number of explicit operations tracked by `File` is reduced only to
  the ones that could interact. All other ops are spawned on the
  blocking pool without being tracked by the `File` instance.

* The `seek` implementation is not backed by a trait and `poll_seek`
  function. This avoids the question of how to model non-blocking seeks
  on top of a blocking file. In this patch, `seek` is represented as an
  `async fn`. If the associated future is dropped before the caller
  observes the return value, we make no effort to define the state in
  which the file ends up.
2019-08-27 12:25:20 -07:00
Carl Lerche 08099bb2d3 net: rewrite TcpStream::connect with async fn (#1497)
This also removes `TcpStream::connect_std` as the conversion functions
from `std` need to be rethought. A note tracking this has been added
to #1209.
2019-08-27 12:08:28 -07:00
Newton Ni 807d536846 codec: fix infinite loop in tokio_codec::LinesCodec (#1489) 2019-08-26 13:38:52 -07:00
Danny Browning 654f9d703f tokio: expose signal feature (#1491)
Expose tokio_net::signal::ctrl_c via tokio::net::signal::ctrl_c as feature signal.
2019-08-22 09:12:00 -07:00
Jon Gjengset a285689664 net: shutdown TCP write when asked to shut down (#1488) 2019-08-21 10:41:51 -07:00
Gurwinder Singh 13930eff2a chore: two async_await feature remained (#1486) 2019-08-21 18:53:42 +09:00
Taiki Endo 24fb33e012 io: add AsyncReadExt::{chain, take} (#1484) 2019-08-20 20:09:07 -07:00
Taiki Endo a791f4a758 chore: bump to newer nightly (#1485) 2019-08-20 20:07:16 -07:00
Eliza Weisman 7e7a5147a3 executor: switch from log to tracing (#1454)
## Motivation

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime. Updating the executor crate is an obvious
starting point. 

## Solution

This branch replaces the use of `log` in `tokio-executor` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation. I've also added a few `tracing` spans, primarily in
places where a variable is added to all the log messages in a scope.

## Notes

For users who are using the legacy `log` output, there is a feature flag
to enable `log` support in `tracing`. I thought about making this on by
default, but that would also enable the `tracing` dependency by default,
and it is only pulled in when the `threadpool` feature flag is enabled.
The `tokio` crate could enable the log feature in its default features
instead, since the threadpool feature is on by default in `tokio`. If
this isn't the right approach, I can change how `log` back-compatibility
is enabled.

We might want to consider adding more `tracing` spans in the threadpool
later. This could be useful for profiling, and for helping users debug
the way their applications interact with the executor. This branch is
just intended as a starting point so that we can begin emitting
`tracing` data from the executor; we should revisit what instrumentation
should be exposed, as well.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-20 12:44:26 -07:00
Jakub Beránek 2d56312b89 timer: introduce delay function shortcut (#1440)
This commit adds a simple delay shortcut to avoid writing Delay::new
everywhere and removes usages of Delay::new.
2019-08-20 08:39:55 -07:00
Ivan Petkov 357df38861 process: move into the tokio-net crate (#1475) 2019-08-19 19:42:54 -07:00
John-John Tedro 34a9dc2d76 Implement FusedStream and FusedFuture for Interval and Delay (#1476) 2019-08-19 10:21:34 -04:00
Ivan Petkov 68d5fcb8d1 docs: fix all rustdoc warnings (#1474) 2019-08-18 14:38:54 -07:00
Ivan Petkov 08b07afbd9 signal: remove new() constructors in favor of free functions (#1472)
* Also removed any `*_with_handle` related methods in favor of always
using the default reactor
2019-08-18 14:22:09 -07:00
Douman 7b0c60849c net: make default reactor guard public (#1468) 2019-08-18 11:36:21 -07:00
Ivan Petkov 6d8d388dc5 docs: add docs.rs metadata to build with all features (#1471) 2019-08-18 11:11:46 -07:00
Ivan Petkov bc61bd9d3d ci: ensure all tests are run for each feature (#1470)
* This includes running docs, examples, and lib tests for each added
feature, to ensure nothing is broken
2019-08-18 10:50:38 -07:00
Jakub Beránek a9585f0318 tokio-process: change CommandExt to a fully asynchronous Command struct (#1448)
Refs: #1371
2019-08-18 10:13:37 -07:00
Carl Lerche 88b4ec84d7 chore: prepare 0.2.0-alpha.2 release (#1465) 2019-08-17 23:34:25 -07:00
Philip Kannegaard Hayes 9f0daad5ac sync: fix fuzz_oneshot test by using instrumented loom::sync::Arc (#1464)
Since `tokio_sync::oneshot` makes a `CausalCell::with_mut()` mutable
access in the `Inner::drop()`, we must use the instrumented
`loom::sync::Arc`.

Uncovered by carllerche/loom#42
2019-08-17 21:30:31 -07:00
Carl Lerche c187cd75b6 signal: move into tokio-net (#1463) 2019-08-17 13:43:55 -07:00
Carl Lerche a83f5e4ba6 uds: move into tokio-net (#1462) 2019-08-16 14:42:05 -07:00
Carl Lerche 4935aae164 udp: remove files left over from moving tokio-udp (#1461) 2019-08-16 11:05:50 -07:00
Carl Lerche ba1829fd26 chore: rename ui-tests -> build-tests (#1460) 2019-08-16 09:26:56 -07:00
Carl Lerche ce7e60e396 udp: move tokio-udp into tokio-net (#1459) 2019-08-16 07:26:10 -07:00
Ivan Petkov d8b23ef852 signal: rename SignalKind methods (#1457)
This renames the SignalKind constructors to be a bit more readable
instead of using the signal names themselves
2019-08-15 21:09:09 -07:00
Carl Lerche 4788d3a9e3 tcp: move tokio-tcp into tokio-net (#1456) 2019-08-15 20:37:25 -07:00
Carl Lerche f1f61a3b15 net: reorganize crate in anticipation of #1264 (#1453)
Space is made to add `tcp`, `udp`, `uds`, ... modules.
2019-08-15 15:04:21 -07:00
Jakub Beránek d0a8e5d6f2 tokio-fs: rewrite std echo example using async/await (#1442)
This PR fixes the echo example in tokio-fs.

Refs: #1255
2019-08-15 13:10:17 -07:00
Carl Lerche 3b27dc31d2 threadpool: move threadpool into tokio-executor (#1452)
The threadpool is behind a feature flag.

Refs: #1264
2019-08-15 13:09:02 -07:00
Douman 37131b2114 runtime: refactor thread-local setters (#1449) 2019-08-15 13:00:57 -07:00
Carl Lerche 8538c25170 reactor: rename tokio-reactor -> tokio-net (#1450)
* reactor: rename tokio-reactor -> tokio-net

This is in preparation for #1264
2019-08-15 11:04:58 -07:00
Jakub Beránek 7b6438a172 tokio: rewrite print_each_packet example using async/await (#1446)
This PR fixes the print each packet example in tokio.

Refs: #1201
2019-08-15 10:42:34 -07:00
Carl Lerche 9de7083be8 executor: move current-thread into crate (#1447)
The `CurrentThread` executor is exposed using a feature flag.

Refs: #1264
2019-08-15 09:52:25 -07:00
John Doneth 8d55f98f6f udp: update tokio_udp::UdpFramed to std::future (#1370) 2019-08-14 11:18:21 -07:00
Taiki Endo 999a600494 io: add async BufReader/BufWriter (#1438) 2019-08-14 10:24:07 -07:00
Ilya Lakhin fb9809c068 executor, threadpool: forward port fix from #1155 (#1433)
Add executor::exit, allowing other executors inside threadpool::blocking.
2019-08-13 21:12:49 -07:00
Douman 517162792f macros: upgrade syn/quote (#1432) 2019-08-13 21:11:26 -07:00
Geoff Shannon fe90d61446 test: add a block_on function to tokio-test (#1431) 2019-08-13 21:10:26 -07:00
Ivan Petkov 338b37884a signal: Add SignalKind for registering signals more easily (#1430)
This avoids having consumers import libc for common signals, and it
improves discoverability since users need not be aware that libc
contains all supported constants.
2019-08-13 21:07:22 -07:00
Ivan Petkov 513326e01d signal: remove driver task for Windows event implementation (#1429)
Windows guarantees handler routines are always invoked in a new thread
(https://docs.microsoft.com/en-us/windows/console/handlerroutine), so we
don't need to use the handler-wake-another-driver technique used in the
Unix implementation

By broadcasting the event notifications from the handler, we no longer
need the Driver task to be spawned, which fixes the starvation issue if
the executor which runs the Driver task goes away

Also changed the behavior so that the default event handler runs if
all listeners for CTRL_{C, BREAK} events go away.
2019-08-13 21:01:06 -07:00
Ivan Petkov 73a91ad7b3 signal: delete blocking Read/Write impls on ChildStd{in, out, err} (#1428) 2019-08-13 20:53:02 -07:00
Taiki Endo 930cce8677 chore: update futures-preview to 0.3.0-alpha.18 (#1427) 2019-08-10 14:09:28 -07:00
Taiki Endo 6a125082e4 chore: apply unreachable_pub and missing_debug_implementations to all crates (#1424) 2019-08-11 04:28:52 +09:00
Taiki Endo d9f9c5658f chore: bump to newer nightly (#1426) 2019-08-11 02:01:20 +09:00
Taiki Endo fff39c03b1 ci: deny warnings in cirrus (#1425) 2019-08-11 01:41:51 +09:00
Tomasz Miąsko 756606a58b uds: implement split and split_mut for UnixStream (#1395)
This mirrors split API available in TcpStream.
2019-08-09 12:50:18 -07:00
Ran Benita e3b4c99a33 codec: a few suggestions (#1418)
How the buffer is managed is often critical for performance. Not
taking care of it will be catastrophic for performance beyond the
initial buffer size with the current implementation (a loop of
`reserve(1)`).
2019-08-09 12:20:31 -07:00
Taiki Endo 42fa0c28d3 timer: use std::sync::atomic::AtomicU64 instead of own AtomicU64 (#1421) 2019-08-10 03:42:03 +09:00
Taiki Endo f7b41c9dcc macros: improve error messages (#1420) 2019-08-09 10:28:22 -07:00
Taiki Endo 73102760cf chore: change default lint level to warning and deny warnings in CI (#1416) 2019-08-10 00:07:57 +09:00
Douman 18833a8e67 macros: Error on function with arguments (#1419) 2019-08-09 11:04:41 -04:00
tmiasko eba8bf2b4b io: implement AsyncWrite for Vec<u8> (#1409) 2019-08-08 20:55:27 -07:00
David Kellum 790d649dc5 update (dev dep) env_logger to latest 0.6 (#1390) 2019-08-08 20:37:32 -07:00
Lucio Franco 50e5d401df chore: prepare for v0.2.0-alpha.1 release (#1410) 2019-08-08 12:48:53 -07:00
Carl Lerche 2e69f2a7fd sync: track upstream loom changes (#1407) 2019-08-07 23:24:22 -07:00
Carl Lerche 962521f449 chore: enable full CI run (#1399)
* update all tests
* fix doc examples
* misc API tweaks
2019-08-07 20:02:13 -07:00
Carl Lerche 831be9c08e executor: remove unused dependency (#1406) 2019-08-07 19:55:42 -07:00
Carl Lerche 23c380a78f sync: track loom changes (#1405) 2019-08-07 15:38:34 -07:00
Lucio Franco 0a05332648 Remove git dep and add macro examples (#1404)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 15:02:38 -07:00
Lucio Franco 7268b0bb3a Migrate threadpool to futures-util (#1403)
* Migrate threadpool to futures-util

Signed-off-by: Lucio Franco <[email protected]>

* fmt
2019-08-07 16:26:44 -04:00
Lucio Franco 6412389bba executor: update park implementation (#1402)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 13:01:13 -07:00
tmiasko 53a94c025d io: implement AsyncBufRead for &[u8] and Cursor (#1397)
* `impl AsyncRead for &[u8]`
* `impl AsyncBufRead for &[u8]`
* `impl<T: AsRef<[u8]> + Unpin> AsyncRead for Cursor<T>`
* `impl<T: AsRef<[u8]> + Unpin> AsyncBufRead for Cursor<T>`
2019-08-07 12:57:37 -07:00
Gurwinder Singh 7174c63bf9 codec: move length delimited codec to tokio-codec (#1401) 2019-08-07 12:10:05 -07:00
Ivan Petkov cb2336ff3d process: Misc polish (#1400)
* Denied all warnings in tests, and denied rust_2018_idioms violations
* Bumped the crate version and set publish = false
* Pruned dependencies:
 - Only pull in tokio-sync on windows where it is used
 - Removed unused dev-dependencies
* Switch to Async{Read, Write} traits from tokio-io rather than
futures-io
* Use #[tokio::test] where possible
* Removed deprecated items
* Fix all doc examples
2019-08-07 10:38:45 -07:00
Carl Lerche 47e2ff48d9 tokio: fix API doc examples (#1396) 2019-08-06 14:03:49 -07:00
Carl Lerche 2f43b0a023 sync: polish and update API doc examples (#1398)
- Remove `poll_*` fns from some of the sync types.
- Move `AtomicWaker` and `Lock` to the root of the `sync` crate.
2019-08-06 13:54:56 -07:00
Carl Lerche 05d00aebb7 uds: remove poll_* fns in favor of async fns (#1394) 2019-08-05 15:05:02 -07:00
Carl Lerche 62733a6594 udp: remove poll_* fns in favor of async fns (#1393)
This removes the need for manual futures.
2019-08-05 14:18:18 -07:00
Carl Lerche 6d8cc4e475 tcp: update API documentation (#1392) 2019-08-05 11:50:55 -07:00
Carl Lerche 6cbe3d4f82 fs: use async fn instead of custom futures (#1381)
Also update all the doc examples.
2019-08-04 11:24:30 -07:00
Carl Lerche 337646b97f tokio: re-export future/stream utils (#1387) 2019-08-03 21:08:29 -07:00
Taiki Endo 0bb015588a codec: add AsyncBufRead/BufRead implementations (#1385)
* AsyncBufRead for FramedWrite2<T>
* BufRead for FramedWrite2<T>
* AsyncBufRead for Fuse<T, U>
* BufRead for Fuse<T, U>
2019-08-03 20:15:50 -07:00
Steven Fackler 63377e2110 Add AsyncWriteExt::shutdown (#1382) 2019-08-03 00:51:24 -04:00
Carl Lerche 878503f965 docs: update API documentation for some crates (#1380)
Updates API documentation for

- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
2019-08-02 14:35:32 -07:00
Carl Lerche 2c01b3e0e0 io: remove util from default features (#1379)
Sub-crates should require opting into features.
2019-08-02 12:59:24 -07:00
Carl Lerche ee9105d166 tokio: add async io traits to prelude (#1378) 2019-08-02 12:50:40 -07:00
Lucio Franco 5a4f849bba tokio: update tinyhttp example to async/await (#1372) 2019-08-02 12:24:15 -07:00
Lucio Franco ff41108834 io: move io helpers back into tokio-io (#1377)
Utilities are made optional with a feature flag.
2019-08-02 12:23:44 -07:00
Lucio Franco 6b202722ea io: Add AsyncWriteExt::flush (#1376)
* io: Add `AsyncWriteExt::flush`

* fmt

* fix clippy
2019-08-02 13:53:49 -04:00
Lucio Franco 144d980e5c tokio: update connect to async/await (#1375) 2019-08-02 10:03:06 -07:00
Lucio Franco 81d789b88f tokio: Update proxy to async/await (#1373) 2019-08-01 20:03:34 -07:00
Lucio Franco 634c19582f chore: add rust-toolchain file to track nightly version (#1374) 2019-08-01 20:00:55 -07:00
Ivan Petkov ff922bbe6d signal: Change constructors to return a result instead of lazy future (#1340) 2019-07-30 18:23:26 -07:00
Gurwinder Singh bf38631d6a chore: Fix spelling mistake (#1359) 2019-07-30 10:53:11 -07:00
Taiki Endo 6dda866191 tokio: re-enable StreamExt (#1362) 2019-07-30 09:55:34 -07:00
Taiki Endo 03e450deb1 sync: switch branch of loom dev-dependency to master (#1367)
* sync: switch branch of loom dev-dependency to master

* replace loom::fuzz with loom::model
2019-07-30 10:11:46 -04:00
andy finch fbf90e6356 Update process to use std::future (#1343) 2019-07-29 18:36:11 -07:00
Shell Chen 74168ae82f tcp: add async fn TcpStream::peek (#1360)
* tcp: add `async fn TcpStream::peek`

* tcp: apply rustfmt on tests
2019-07-26 10:55:02 -04:00
John Doneth d038009e7d Update chat example to async/await (#1349) 2019-07-25 19:44:23 -04:00
John Doneth 132e9f1da5 Update examples to return Result (#1305)
* update echo-udp

* update echo

* update hello_world

* update udp-client

* rustfmt

* remove send & sync

* rebase & change new updated examples
2019-07-25 16:47:31 -04:00
Taiki Endo fe021e6c00 ci: enable clippy lints (#1335) 2019-07-26 03:47:14 +09:00
Lucio Franco f311ac3d4f buf: Inital pass at updating BufStream (#1355) 2019-07-25 14:21:48 -04:00
Shell Chen 298be80249 tokio: include async-trait feature for uds (#1352) 2019-07-25 08:07:33 -07:00
John Doneth 79b017c773 Export LinesCodecError (#1350) 2019-07-24 15:26:41 -04:00
Taiki Endo ca0e5cc670 add TryFrom/From implementations (#1347)
* TryFrom<net::TcpListener> for TcpListener
* TryFrom<net::TcpStream> for TcpStream
* TryFrom<net::UdpSocket> for UdpSocket
* TryFrom<net::UnixDatagram> for UnixDatagram
* TryFrom<net::UnixListener> for UnixListener
* TryFrom<net::UnixStream> for UnixStream
* TryFrom<UnixDatagram> for mio_uds::UnixDatagram
* TryFrom<File> for io::File
* From<io::File> for File
2019-07-24 09:26:12 -07:00
Douman 59bc364a0e macros: detect double test attribute (#1336) 2019-07-22 09:28:07 -07:00
Taiki Endo e88d10a3cb chore: bump to newer nightly (#1338) 2019-07-22 06:04:02 +09:00
Ivan Petkov a3b8d82711 Merge tokio-process into tokio
Original repo can be found at https://github.com/alexcrichton/tokio-process/
2019-07-21 11:08:16 -07:00
Ivan Petkov d9688bc094 signal: change unix::Signal to return () instead of signum (#1330)
* This simplifies the API surface by returning () instead of the signal
number that was used during registration. This also more closely mirrors
the cross-platform `CtrlC` event stream API
* This is a **breaking change**
2019-07-20 15:12:53 -07:00
Ivan Petkov 320a5fdca7 signal: replace windows::Event with windows::CtrlBreak (#1331)
* Add a new `windows::CtrlBreak` struct which wil represent a stream of
CTRL_BREAK_EVENT signals on Windows systems
* The `windows::Event` type is no longer publicly accessible and is
replaced by using `CtrlC` or `windows::CtrlBreak`.

[breaking-change]
2019-07-20 10:50:27 -07:00
Taiki Endo 9af07ce208 chore: remove redundant field names in struct literals (#1334) 2019-07-20 10:43:19 -07:00
Taiki Endo 1b2d997863 chore: use ptr::{null, null_mut} instead of 0 as *{const, mut} (#1333) 2019-07-20 10:41:02 -07:00
Taiki Endo 7a52ddcd09 chore: remove unnecessary conversion (#1332) 2019-07-20 12:10:42 -04:00
Carl Lerche 9d3e5aac08 tokio: remove Send + 'static requirement from block_on (#1329)
Removes the `Send` requirement to futures passed to `Runtime::block_on`.
Previously, `block_on` was implemented by sending the future to a
runtime thread. In order to do this, the future must be Send.

The reason why the future is sent to the pool is because we cannot
guarantee, while off the pool, that a reactor / timer thread is running.
This is due to a limitation in the current version of tokio-threadpool.
There is a plan to fix this (#1177), but the proper fix is non trivial.

In order to unblock APIs that require this, this patch updates the
runtime to spawn an always running thread containing a reactor and
timer. All calls to `block_on` will use that reactor and timer.
2019-07-19 17:25:04 -07:00
Carl Lerche a99fa6e096 chore: remove tokio-futures facade crate (#1327)
This switches from using the tokio-futures facade to referencing
futures-* crates directly.
2019-07-19 13:11:46 -07:00
David Kellum b89ed00a0d Remove last non-dev dependency on rand crate (#1324)
Use std RandomState for XorShift seeding. This allows dropping _rand_
crate dep here, accept as a dev dependency for tests or benchmarks.
2019-07-19 12:12:32 -07:00
Dylan Frankland 12ce75f088 fs: add remove_dir_all and RemoveDirAllFuture (#1325)
Adds the sister function to `remove_dir` and mirrors the `create_dir_all` that's already exposed.
2019-07-19 12:09:53 -07:00
Taiki Endo a88308ed9f tokio: add AsyncReadExt::read_to_string (#1326) 2019-07-19 11:50:00 -07:00
João Oliveira a298472da8 tokio-tls: enable Send and Sync (#1317)
-  update 0 as *mut () calls to std::ptr::null_mut()
- impl Send and Sync for AllowStd
2019-07-19 10:21:26 -07:00
Shell Chen d0bb16192b timer: change Into to From trait for Elapsed (#1322) 2019-07-17 09:04:59 -04:00
Shell Chen a18ddb3b61 timer: impl Into<std::io::Error> for Elpased (#1321)
That convert Elpased to ErrorKind::TimedOut
2019-07-16 20:22:03 -07:00
Jon Gjengset 003b4d8074 Get rid of Enter for with_default (#1315)
We want executors to enforce that there are never multiple active at the
same time. This is ensured through `Enter`, which will panic if you
attempt to create more than one. However, by requiring you to pass an
`&mut Enter` to `executor::with_default`, we were *also* disallowing
temporarily overriding the current executor.

This patch removes that requirement.
2019-07-16 14:29:35 -04:00
Yin Guanhao 6d186fe40e Replace (some) uninitialized with MaybeUninit (#1295) 2019-07-16 10:47:46 -07:00
Diggory Blake 0d99ddd4f4 tcp: implement "split_mut" for TcpStream (#1289) 2019-07-16 10:28:00 -07:00
João Oliveira 448d9d2eab tls: update to std-future (#1224) 2019-07-16 10:26:08 -07:00
David Kellum 0de3a69eb4 fs: drop deprecated tempdir crate use in tests (#1312)
In particular because it pulls in old rand duplicates. Replace use
with tempfile::tempdir() which has been available since tempfile
3.0.0.
2019-07-15 15:32:33 -07:00
John Doneth 61aee5fc28 examples: pdate tinydb example (#1288)
Update tinydb example to use async / await.
2019-07-15 15:19:36 -07:00
Sean McArthur 7f7f74985e io: Minor adjustments to tokio-test IO (#1306)
This also re-exports `bytes::{Buf, BufMut}` from `tokio-io`.
2019-07-15 14:53:16 -07:00
Jon Gjengset e6cf976662 tokio: include async-traits feature (#1314)
The `tokio` facade crate will depend on the `async-traits` feature flag in
sub crates.
2019-07-15 14:02:14 -07:00
Taiki Endo b14e189e44 add #[must_use] to more futures and streams (#1309) 2019-07-15 13:28:56 -07:00
Taiki Endo 2dde2b448f Fix import of ready macro 2019-07-15 11:52:13 -07:00
Taiki Endo 6742816e78 tokio: add AsyncBufReadExt::lines 2019-07-15 11:52:13 -07:00
Taiki Endo ab040bb498 tokio: add AsyncBufReadExt::read_line 2019-07-15 11:52:13 -07:00
Taiki Endo 0cfa120ba8 tokio: add AsyncBufReadExt::read_until 2019-07-15 11:52:13 -07:00
Taiki Endo 5774a9cd64 io: add AsyncBufRead trait 2019-07-15 11:52:13 -07:00
John Doneth da49ede41e update udp-codec example (#1293) 2019-07-15 14:14:03 -04:00
Carl Lerche d224d6415e chore: indicate the master branch docs are old. (#1304)
Fixes #1292
2019-07-15 10:44:47 -07:00
Gurwinder Singh 83273b8b50 chore: use ready macro from futures-core (#1300) 2019-07-15 10:43:54 -07:00
Taiki Endo ca708d6d87 chore: update rand dependency to 0.7 (#1302) 2019-07-15 10:13:10 -07:00
matthieugras 0b75c0c53d executor: block thread when needed in block fn (#1303)
Fix #1296
2019-07-15 08:56:20 -07:00
Alex Gaynor 5fbb36a060 reactor: bump parking_lot dependency (#1298) 2019-07-14 09:28:54 -07:00
Gurwinder Singh c897a5b696 Re-export tokio-fs (#1287) 2019-07-14 12:03:49 -04:00
Sean McArthur 48d7f7b931 tokio-test: add tokio_test::io mock builder 2019-07-12 11:19:12 -07:00
Carl Lerche 2291823181 tokio: re-export correct tokio-uds version (#1286)
An earlier PR (#1282) re-exported the version from crates.io and not git
master.
2019-07-11 10:27:51 -07:00
andy finch 795e02f4c6 fs: update to use std::future (#1269) 2019-07-11 09:05:49 -07:00
Carl Lerche 7ac8bfc821 chore: bump to newer nightly (#1284) 2019-07-10 14:36:36 -07:00
Carl Lerche a79483750f tokio: update echo example (#1283) 2019-07-10 14:21:20 -07:00
Carl Lerche 3855f373d3 tokio: re-export tokio-uds (#1282)
The tokio-uds crate has been previously updated to std::future. This
commit enables the re-export in the tokio facade crate.
2019-07-10 11:21:27 -07:00
Carl Lerche bd3f3270db tokio: update threaded runtime to std::future (#1280)
re-enables the threaded runtime and sets it (again) as the default.
2019-07-10 11:21:06 -07:00
Taiki Endo e5525628cd chore: remove usage of deprecated ONCE_INIT (#1281) 2019-07-10 08:35:07 -07:00
Carl Lerche f1b8a318d9 tokio: add AsyncReadExt::read_to_end (#1279) 2019-07-09 16:17:58 -07:00
Carl Lerche 64343f1b78 tokio: add AsyncWriteExt::write_all (#1277) 2019-07-09 12:37:14 -07:00
Ruben De Smet 82795184c1 tokio: rewrite examples with async. (#1228) 2019-07-09 11:21:12 -07:00
Thomas Lacroix f529928d87 chore: script updating versions in links to docs.rs (#1249) 2019-07-09 11:19:38 -07:00
Ivan Petkov 461eebe612 signal: Replace ctrl_c with a CtrlC struct (#1273)
* Add a new `CtrlC` struct which will represent a stream of SIGINT
signals on Unix or the CTRL_C event on Windows
* `CtrlC` implements `Stream<Output = ()>` rather than `IoSteam` as
previously
2019-07-09 08:48:46 -07:00
Gurwinder Singh 407d15cf93 chore: Add link to docs (#1276) 2019-07-09 11:21:29 -04:00
Yin Guanhao 80915906d8 uds: update to std-future (#1227) 2019-07-08 14:58:40 -07:00
Yin Guanhao 88e775dcf0 udp: UdpSocket split support (#1226) 2019-07-08 14:47:31 -07:00
Carl Lerche 8b49a1e05f chore: update examples link in README (#1274) 2019-07-08 13:34:39 -07:00
Carl Lerche 8fa1510d67 timer: fix build (#1275) 2019-07-08 11:23:15 -07:00
Reto Kaiser 7797a377c3 current-thread: make tokio_current_thread::Handle Sync (#1119) 2019-07-08 10:25:34 -07:00
Steven Fackler b62d224fac timer: fix Handle::timeout (#1093)
The old implementation didn't work for Timeout<Stream>, since the method
took a deadline rather than a timeout.
2019-07-08 10:18:37 -07:00
Aaron Hill d4803bc868 Use Sink trait from futures-sink-preview (#1244) 2019-07-08 09:56:11 -07:00
Thomas Lacroix e07a03b3c5 signal: update instructions in Ctrl-C example (#1270)
Fixes: #1248
2019-07-07 09:49:19 -07:00
Taiki Endo 7b86acb71d chore: Update futures-preview to 0.3.0-alpha.17 (#1267) 2019-07-04 14:34:57 -07:00
Steffen Butzer 0651f09427 Remove usage of deprecated std::error::Error methods (#1206) (#1245) 2019-07-03 23:06:03 -07:00
Thomas Lacroix 516251052d Add missing links in README.md (#1233)
Fixes: #1229
2019-07-03 22:59:10 -07:00
Ivan Petkov cbad83f362 signal: migrate to std::futures (#1218)
Migrate to std::futures and the futures 0.3 preview and use async/await
where possible

**Breaking change:** the IoFuture and IoStream definitions used to refer
to Box<dyn Future> and Box<dyn Stream>, but now they are defined as
Pin<...> versions which are technically breaking.

No other breaking or functional changes have been made
2019-07-03 10:40:59 -07:00
Eliza Weisman bd9760e124 add release documentation to CONTRIBUTING.md (#1171)
## Motivation

Currently, the process for releasing a new version of a Tokio crate is
somewhat complex, and is not well-documented. To make it easier for
contributors to release minor versions more frequently, there should be
documentation describing this process.

## Solution

This branch adds a section to `CONTRIBUTING.md` describing how to
release a new version of a Tokio crate. The steps are based on those
described by @carllerche in an offline conversation.

I've also added a quick shell script to actually publish new crate 
versions. This should make it harder to make mistakes when 
publishing.

Signed-off-by: Eliza Weisman <[email protected]>
2019-07-03 10:18:02 -07:00
Carl Lerche 3e898f58a5 tcp: add ascyc fn TcpListener::accept (#1242)
Refs: #1209
2019-07-03 09:49:56 -07:00
Ivan Petkov c531865d2c ci: don't generate docs for deps on FreeBSD (#1241) 2019-07-03 09:41:35 -07:00
Ivan Petkov 722eb257be ci: scope each tests/examples invocation to a specific crate (#1238) 2019-07-03 08:49:05 -07:00
Taiki Endo ceed29586b io: fix documents (#1231) 2019-07-01 20:44:12 -07:00
Carl Lerche 70eca184f0 tokio: re-enable timer in runtimes (#1237)
This also brings back the timer tests in the tokio crate.
2019-07-01 18:27:13 -07:00
Carl Lerche b2c777846e timer: finish updating timer (#1222)
* timer: restructure feature flags
* update timer tests
* Add `async-traits` to CI

This also disables a buggy `threadpool` test. This test should be fixed in the future.

Refs #1225
2019-06-30 08:48:53 -07:00
Lucio Franco 8e7d8af588 docs: add note in the readme about the master branch (#1230) 2019-06-29 21:47:20 -04:00
Yin Guanhao 7380dd2482 TcpSocket specialized split (#1217) 2019-06-28 23:36:49 -07:00
Eliza Weisman af46eac583 chore: remove tokio-trace, add "Related Projects" to README (#1221)
## Motivation

The `tokio-trace` and `tokio-trace-core` crates have been renamed to
`tracing` and `tracing-core`, and moved to their own repository
(`tokio-rs/tracing`).

## Solution

This branch removes `tokio-trace` and `tokio-trace-core` from the
`tokio` repository. In addition, I've added a "Related Projects" section
to the root README, which lists `tracing` (as well as  `mio`, and
`bytes`) as other libraries maintained by the Tokio project. I thought
that this would help folks looking for `tokio-trace` here find it in its
new home.

In addition, it changes `tokio` to depend on `tracing-core` rather than
`tokio-trace-core`.

Closes #1159

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-28 13:13:46 -07:00
Carl Lerche e7488d983e threadpool: update to std::future (#1219)
An initial pass at updating `tokio-threadpool` to `std::future`. The
codebase and tests both now run using `std::future` but the wake
mechanism is not ideal. Follow up work will be required to improve on
this.

Refs: #1200
2019-06-27 22:30:56 -07:00
Sean McArthur e4415d986a sync: change oneshot poll_close to poll_closed
The action of `Sender::poll_close` is to check if the receiver has been
closed, not to try to close the sender itself. So change to
`poll_closed`.
2019-06-27 13:56:58 -07:00
Carl Lerche ff906acdfb ci: disable cache on cirrus (#1215)
Caching takes longer than rebuilding
2019-06-27 12:08:43 -07:00
Carl Lerche 32ceccb465 sync: add async APIs to oneshot and mpsc (#1211)
Adds:

- oneshot::Sender::close
- mpsc::Receiver::recv
- mpsc::Sender::send

Also renames `poll_next` to `poll_recv`.

Refs: #1210
2019-06-27 11:33:36 -07:00
Douman 0af05e7408 macros: allow configuring runtime used by main macro (#1185) 2019-06-27 10:40:21 -07:00
jesskfullwood 6b9e7bdace codec: update to use std-future (#1214)
Strategy was to

- copy the old codec code that was temporarily being stashed in `tokio-io`
- modify all the type signatures to use Pin, as literal a translation as possible
- fix up the tests likewise

This is intended just to get things compiling and passing tests. Beyond that there is surely
lots of refactoring that can be done to make things more idiomatic. The docs are unchanged.

Closes #1189
2019-06-27 10:10:29 -07:00
Carl Lerche ed4d4a5353 chore: format code and enable rustfmt CI task (#1212) 2019-06-27 00:05:01 -07:00
Carl Lerche 1f47ed3dcc tokio: rewrite io_read.rs test to use async/await (#1207)
This simplifies the test
2019-06-26 17:06:56 -07:00
Carl Lerche e9aaacddbd tokio: re-export sync::{lock,mpsc} (#1208)
These types have been updated already.
2019-06-26 16:54:15 -07:00
Carl Lerche 11f6b2862f tokio: move I/O helpers to ext traits (#1204)
Refs: #1203
2019-06-26 14:42:19 -07:00
Carl Lerche 8404f796ac test: get cargo test --tests working (#1205)
Broken tests are disabled
2019-06-26 14:40:52 -07:00
Yin Guanhao 6316aa1d0b Update tokio-udp to use std-future (#1199) 2019-06-26 14:41:36 -04:00
Bhargav 0784dc2767 tokio: add read_exact method (#1202) 2019-06-26 11:36:09 -07:00
Denis dd126c2333 Implement TryFrom to transform various I/O primitives into their mio counterparts (#1158)
* `TryFrom<TcpListener> for mio::net::TcpListener`
* `TryFrom<TcpStream> for mio::net::TcpStream`
* `TryFrom<UdpSocket> for mio::net::UdpSocket`
* `TryFrom<UnixListener> for mio_uds::UnixListener`
* `TryFrom<UnixStream> for mio_uds::UnixStream`
2019-06-26 08:51:38 -07:00
Lucio Franco 3cc33dca7c sync: Fix lock test to actually test the inner lock value (#1197)
* sync: Fix lock test to actually test the returned value

* Update lock test to use task.poll
2019-06-26 11:32:41 -04:00
Carl Lerche dc5fa80a09 macros: re-export main macro from tokio (#1198)
Includes minor fixes and a very basic example.

Fixes #1183
2019-06-25 20:14:21 -07:00
Zahari Dichev 455782b964 trace: Allow setting event parents explicitly (#1109)
## Motivation 

As mentioned in tokio-rs/tracing#1100  it makes sense to be able to set
the parents of events explicitly.

## Solution 

For that to happen the Parent type is extracted from span.rs and a
`parent` field is added to Event. Additionally the appropriate macros
arms are added with corresponding tests as described in
tokio-rs/tracing#1100

Closes tokio-rs/tracing#1100

Signed-off-by: Zahari Dichev <[email protected]>
2019-06-25 15:12:52 -07:00
Lucio Franco 29e417c257 tokio: Add io copy, read, and write (#1187) 2019-06-25 16:51:49 -04:00
Ivan Petkov 9df1140340 signal: factor out event delivery into its own module to share between Unix and Windows (#1174)
Today the Unix and Windows implementations have similar yet differing
implementations of hooking into OS events and propagating them to any
listening futures. Rather than re-implement the same behavior two
different ways, we should factor out any commonality into a shared
module and keep the Unix/Windows modules focused solely on OS
integrations.

Reusing the same implementation across OS versions also allows for more
consistent behavior between platforms, which also makes squashing bugs
much easier.

This change introduces the `registry` module which handles creating and
initializing a global map of signals/events and their registered
listeners. Each OS specific module is expected to implement the OS hooks
which delegate to invoking the registry module's methods for
distributing the event notifications.

# Use registry module for Windows implementation

Note this still uses the same architecture as previously: a driver task
is spawned by the first registered event, and that task is responsible
for delivering any events to registered futures. (If that first event
loop goes away, all events will deadlock). A solution to this issue will
be explored at a later time.
2019-06-25 13:07:59 -07:00
Lucio Franco e2b4bdb647 sync: Add LockFuture for Lock (#1184) 2019-06-25 10:42:35 -07:00
Ivan Petkov c6defbce4b process: Move files to their own directory 2019-06-24 17:31:47 -07:00
Ivan Petkov b7846a4e2f process: Remove unneeded files 2019-06-24 17:31:00 -07:00
Ivan Petkov cb8607a816 process: Update to 2018 edition 2019-06-24 17:29:33 -07:00
Ivan Petkov 27c15471c1 process: Run cargo fmt 2019-06-24 17:29:33 -07:00
Ivan Petkov 0ab25878bd process: Update README 2019-06-24 17:29:32 -07:00
Eliza Weisman 448302c3d4 trace: Improve documentation (#1148) 2019-06-24 19:22:05 -05:00
Ivan Petkov 934a1467d4 process: Update CHANGELOG 2019-06-24 17:12:17 -07:00
Ivan Petkov 4d639e246b process: Update Cargo.toml 2019-06-24 17:10:58 -07:00
Ivan Petkov ff5381de8d process: Update license files 2019-06-24 17:10:58 -07:00
Ivan Petkov 061452dc01 process: Delete flaky and (now) unused test 2019-06-24 17:10:58 -07:00
Ivan Petkov a6b2682309 process: Bump to 0.2.4 2019-06-24 16:57:20 -07:00
Ivan Petkov cf84a59e5a process: Don't kill child on drop if already successfully killed 2019-06-24 16:57:20 -07:00
Ivan Petkov e90e33d5df process: Add unit tests for dropping killing dropped children 2019-06-24 16:57:20 -07:00
Ivan Petkov fa5da27d98 process: Utilize a global orphan process queue to avoid leaks 2019-06-24 16:57:20 -07:00
Ivan Petkov ecaa069f0f process: Implement a queue for repeatedly attempting to reap orphaned processes 2019-06-24 16:57:20 -07:00
Ivan Petkov fc15d7d4a4 process: Only pull in mio dependency on unix platforms 2019-06-24 16:57:20 -07:00
Ivan Petkov a70a3b599a process: ci: move cargo tool installation to after_success 2019-06-24 16:57:20 -07:00
Ivan Petkov 26faefcc34 process: ci: enable clippy checks as part of the build 2019-06-24 16:57:19 -07:00
Ivan Petkov f16725ea9f process: Fix clippy warnings 2019-06-24 16:57:19 -07:00
Ivan Petkov caf43221b5 process: ci: fix cargo binary caching 2019-06-24 16:57:19 -07:00
Ivan Petkov 93680357dd process: Fix drop_kills test when running on macOS with a single thread 2019-06-24 16:57:19 -07:00
Ivan Petkov 784d21ae31 process: Try pinning mio to 0.1.16 2019-06-24 16:57:19 -07:00
Ivan Petkov 0938ccfefd process: ci: cache cargo tarpaulin build 2019-06-24 16:57:19 -07:00
Ivan Petkov 6fa2fdab44 process: Ensure all tests are run with an explicit timeout 2019-06-24 16:57:19 -07:00
Ivan Petkov d0d13d0bd0 process: Change codecov comment behavior to default 2019-06-24 16:57:19 -07:00
Ivan Petkov 8a1777b800 process: Rename EventedReaper to Reaper 2019-06-24 16:57:18 -07:00
Ivan Petkov 42d0f53ddb process: Optimize out the "reaped" flag 2019-06-24 16:57:18 -07:00
Ivan Petkov db0c4147c8 process: Refactor Unix process handling 2019-06-24 16:57:18 -07:00
Ivan Petkov 10fd2afd18 process: Simplify child IO registration 2019-06-24 16:57:18 -07:00
Ivan Petkov 83a55601ef process: Move src/unix.rs to src/unix/mod.rs 2019-06-24 16:57:18 -07:00
Ivan Petkov b37120f61c process: Update line-by-line doc example to be more flexible 2019-06-24 16:57:18 -07:00
Ivan Petkov 91dbf24cf4 process: Update min supported rust version as per the Tokio project policy 2019-06-24 16:57:18 -07:00
Ivan Petkov c78fd6d6c5 process: Update Travis link from .org to .com 2019-06-24 16:57:18 -07:00
Ivan Petkov 025474dfbb process: ci: Install cargo-tarpaulin *after* initial tests 2019-06-24 16:57:18 -07:00
Ivan Petkov e7dfcf90fe process: ci: Enable code coverage tracking via codecov.io 2019-06-24 16:57:17 -07:00
Ivan Petkov ecdfe4c474 process: ci: collect code coverage info via cargo-tarpaulin 2019-06-24 16:57:17 -07:00
Ivan Petkov 37b4efb9e2 process: Bump version to 0.2.3 2019-06-24 16:57:17 -07:00
Ivan Petkov c94f607f1b process: Fix some test case deprecation warnings 2019-06-24 16:57:17 -07:00
Ivan Petkov 76438c9e70 process: Implement AsRawHandle for ChildStd{in, out, err} for parity 2019-06-24 16:57:17 -07:00
Yuya Nishihara e0e9594f71 process: Implement AsRawFd for ChildStd* structs 2019-06-24 16:57:17 -07:00
Yuya Nishihara 3b43262a10 process: Implement AsRawFd for inner Fd<T> wrappers and use it instead of self.0 2019-06-24 16:57:17 -07:00
Ivan Petkov 5f18bf669f process: Bump minimum supported rustc version to 1.26 2019-06-24 16:57:17 -07:00
Ivan Petkov 1581c8b475 process: Bump minimum required version of tokio-signal to 0.2.5 2019-06-24 16:57:16 -07:00
Ivan Petkov d3b2efc815 process: Add regression test for signal starvation 2019-06-24 16:56:53 -07:00
Ivan Petkov f7c4e3cd84 process: Bump min supported rustc version to 1.25 2019-06-24 16:56:53 -07:00
Ivan Petkov 329ad3324c process: Bump to 0.2.2 2019-06-24 16:56:53 -07:00
Ivan Petkov 2b6695d25a process: Update CHANGELOG 2019-06-24 16:56:53 -07:00
Ivan Petkov 9290602815 process: Unix: preregister for signal notifications before polling child 2019-06-24 16:56:53 -07:00
Ivan Petkov 827e77e71e process: Bump to 0.2.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 7b3e4b98ac process: Update Child::forget example to use the tokio runtime 2019-06-24 16:56:52 -07:00
Ivan Petkov 5e9d60e834 process: Add a CHANGELOG 2019-06-24 16:56:52 -07:00
Ivan Petkov 8270965459 process: Remove dependency on tokio-core 2019-06-24 16:56:52 -07:00
Ivan Petkov e6b044a820 process: Bump tokio-signal version to 0.2 2019-06-24 16:56:52 -07:00
Ivan Petkov de9b401457 process: Mark status_async2/StatusAsync2 as deprecated 2019-06-24 16:56:52 -07:00
Ivan Petkov ad5179b2d5 process: Remove all items deprecated in 0.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 0aceba21bd process: Bump to 0.1.6 2019-06-24 16:56:52 -07:00
Ivan Petkov 09e21eceea process: Unix: mark child as reaped on kill 2019-06-24 16:56:52 -07:00
Arvid E. Picciani bdc87856f2 process: fix zombification on Drop on unix 2019-06-24 16:56:51 -07:00
Ivan Petkov 7987b64445 process: Clarify that Child::forget docs that it can leak OS resources 2019-06-24 16:56:51 -07:00
Alex Crichton 32c928b607 process: Bump to 0.1.5 2019-06-24 16:56:51 -07:00
Alex Crichton 82aeae147d process: Update dev-dependencies 2019-06-24 16:56:51 -07:00
Alex Crichton f48944c1fb process: Update winapi to 0.3 2019-06-24 16:56:51 -07:00
Ivan Petkov f0680617ee process: Fix project name typo in README 2019-06-24 16:56:51 -07:00
Alex Crichton dbc185cd3a process: Tweak travis config 2019-06-24 16:56:51 -07:00
Alex Crichton c205e2c358 process: Fix copy/paste 2019-06-24 16:56:51 -07:00
Alex Crichton acec6356ee process: Clarify wording of license information in README. 2019-06-24 16:56:51 -07:00
Alex Crichton c11eec3908 process: Bump to 0.1.4 2019-06-24 16:56:50 -07:00
Alex Crichton 69295fac1e process: Add an Errors section to status_async2 2019-06-24 16:56:50 -07:00
Ivan Petkov b9c6eb309c process: Add status_async2 as a closer analog to spawn_async 2019-06-24 16:56:50 -07:00
Ivan Petkov 56d3914675 process: Bugfix: ensure status_async closes child's stdio handles after spawning 2019-06-24 16:56:50 -07:00
Ivan Petkov 34e71fa71a process: Add must_use annotations to all futures 2019-06-24 16:56:50 -07:00
Ivan Petkov 914b803429 process: Add Debug impls for nondeprecated structs 2019-06-24 16:56:50 -07:00
Alex Crichton 4d11784b01 process: Tweak docs and macro imports 2019-06-24 16:56:50 -07:00
Michael Pankov 50cabae181 process: Add an example with reading input line-by-line 2019-06-24 16:56:50 -07:00
Alex Crichton c101e9e11d process: Use appveyor to download rustup 2019-06-24 16:56:49 -07:00
Alex Crichton 5c5f793ef0 process: Bump to 0.1.3 2019-06-24 16:56:49 -07:00
Alex Crichton 1384b31d60 process: Update to tokio-io, mio, and tokio-core changes 2019-06-24 16:56:49 -07:00
Alex Crichton 521dc94021 process: Bump to 0.1.2 2019-06-24 16:56:49 -07:00
Alex Crichton ed23a06fb1 process: Update doc urls and metadata 2019-06-24 16:56:49 -07:00
Alex Crichton 1aee22505a process: Remove caveat about tokio-signal 2019-06-24 16:56:49 -07:00
Alex Crichton 01b5bf6761 process: Use join3 instead of two joins 2019-06-24 16:56:49 -07:00
Alex Crichton 6638cbc80e process: Update README 2019-06-24 16:56:49 -07:00
Alex Crichton 22bc5e2738 process: Bump back to 0.1.1 2019-06-24 16:56:49 -07:00
Alex Crichton a0c162c0ff process: Hide compat from docs 2019-06-24 16:56:48 -07:00
Alex Crichton ca51ae9651 process: Add back in 0.1.0 compatibility layer 2019-06-24 16:56:48 -07:00
Alex Crichton f3f99b723f process: Bump to 0.2.0 2019-06-24 16:56:48 -07:00
Alex Crichton f20e7a4d2b process: Bump minimum version of tokio-core 2019-06-24 16:56:48 -07:00
Alex Crichton 4a92c4d4b6 process: Tweak drop_kills test 2019-06-24 16:56:48 -07:00
Alex Crichton 9680ecc109 process: Share init in tests 2019-06-24 16:56:48 -07:00
Alex Crichton 124391e42b process: Add a simple wait_with_output test 2019-06-24 16:56:48 -07:00
Alex Crichton 6150be189f process: Rewrite the crate with an extension trait 2019-06-24 16:56:48 -07:00
Ivan Petkov ca9586a089 process: Add documentation to public declarations 2019-06-24 16:56:47 -07:00
Ivan Petkov 89b9792931 process: Update README with crates.io info 2019-06-24 16:56:47 -07:00
Alex Crichton a0cc60153a process: Fix nightly tests 2019-06-24 16:56:47 -07:00
Alex Crichton 7f3f868b66 process: Add Windows support for stdio streams 2019-06-24 16:56:47 -07:00
Andreas Rottmann 97ebb2275c process: [WIP] Actually be non-blocking 2019-06-24 16:56:47 -07:00
Andreas Rottmann 849a5ad0b2 process: Add support for stdio streams 2019-06-24 16:56:47 -07:00
Alex Crichton b16a8613b1 process: Test on stable 2019-06-24 16:56:47 -07:00
Alex Crichton 5664660156 process: Fix tests on nightly 2019-06-24 16:56:47 -07:00
Alex Crichton 4416ea07d8 process: Update travis token 2019-06-24 16:56:47 -07:00
Alex Crichton 56222c588b process: pass --target on appveyor 2019-06-24 16:56:46 -07:00
Alex Crichton 72179d49c5 process: Update to crates.io versions of deps 2019-06-24 16:56:46 -07:00
Alex Crichton 073a1a251a process: Track tokio-core master 2019-06-24 16:56:46 -07:00
Alex Crichton 31c81faf96 process: Add appveyor to readme 2019-06-24 16:56:46 -07:00
Alex Crichton 5e68b0d51d process: Don't build on stable, start w/ beta for now 2019-06-24 16:56:46 -07:00
Alex Crichton 4bd07ac6aa process: Add metadata info 2019-06-24 16:56:46 -07:00
Alex Crichton f4f7bb232e process: Fix a test on Windows 2019-06-24 16:56:46 -07:00
Alex Crichton 413e1b78a7 process: Fix a segfault on windows 2019-06-24 16:56:46 -07:00
Alex Crichton 649fa13a15 process: Remove unused imports 2019-06-24 16:56:45 -07:00
Alex Crichton eef655f3b1 process: Add a Windows implementation 2019-06-24 16:56:45 -07:00
Alex Crichton 97508096fa process: Initial commit 2019-06-24 16:56:41 -07:00
Carl Lerche 06c473e628 Update Tokio to use std::future. (#1120)
A first pass at updating Tokio to use `std::future`.

Implementations of `Future` from the futures crate are updated to implement
`Future` from std. Implementations of `Stream` are moved to a feature flag.

This commits disables a number of crates that have not yet been updated.
2019-06-24 12:34:30 -07:00
James Gilles aa99950b9c trace: Switch benchmarks to criterion (#1163)
Extracted from #1152

This makes it possible to run benchmarks on stable + gives more statistical reliability.
2019-06-24 12:05:45 -07:00
Takanori Ishibashi aac6998c22 chore: fix url in docs (#1173) 2019-06-24 07:33:46 -04:00
Matt Bilker df2c3cd475 trace: fix debug and debug_span macro regression from #1103 (#1170)
PR #1103 accidentally changed the log level for the debug and
debug_span macros to use the INFO level instead of the DEBUG
level. This PR corrects this regression back to the intended
behavior.
2019-06-22 16:31:25 -07:00
James Gilles 36ed35c52c trace: add program-wide default dispatcher (#1152)
## Motivation

I was just trying to use tokio-trace for a greenfield project, but I was frustrated to discover that I couldn't really use it easily.

I was using the [`runtime`](https://docs.rs/runtime/0.3.0-alpha.4/runtime/) crate, which transparently spawns a thread pool executor for futures. In that thread pool, there's no way to set a tokio-trace subscriber for the duration of each thread, since you don't control the thread initialization. You *might* be able to wrap every future you spawn with a subscriber call, but that's a lot of work.

I was also confused because the documentation said that setting a subscriber in the main thread would use that subscriber for the rest of the program. That isn't the case, though -- the subscriber will be used only on the main thread, and not on worker threads, etc.

## Solution

I added a function `set_global_default`, which works similarly to the `log` crate:

```rust
tokio_trace::subscriber::set_global_default(FooSubscriber::new());
```

The global subscriber (actually a global `Dispatch`) is a `static mut` protected by an atomic; implementation is copied from the `log` crate. It is used as a fallback if a thread has no `Dispatch` currently set. This is extremely simple to use, and doesn't break any existing functionality.

Performance-wise, thread-local `Dispatch` lookup goes from ~4.5ns to ~5ns, according to the benchmarks. So, barely any runtime overhead. (Presumably there's a little compile-time overhead but idk how to measure that.) Since the atomic guard is only ever written once, it will be shared among a CPU's cores and read very cheaply.

I added some docs to partially address #1151. I also switched the tokio-trace benchmarks to criterion because the nightly benchmarks weren't compiling (missing `dyn` flags?)
2019-06-21 16:49:53 -07:00
Eliza Weisman 5925ca7720 trace: fix level_span macros not propagating parents (#1167)
Currently, when the `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!`, and `error_span!` macros are invoked with an explicit
parent, a name, and zero or more fields (no target), the macros don't
pass along the explicitly provided parent when expanding to the `span!`
macro. This is likely due to an oversight on my part.

This branch fixes these macros by adding the parent into the `span!`
macro expansion. I've also added a test to catch regressions

Shoutout to @jonhoo for catching this one!

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-21 11:17:06 -07:00
Max Bruckner 2ac132fb46 runtime: better error message in block_on_all on panics (#1166) 2019-06-21 11:10:58 -04:00
Hung-I Wang f9a0cb8792 timer: Implement Default for DelayQueue (#1118) 2019-06-21 10:42:52 -04:00
Igor Gnatenko 9fa6092e5a chore: Update parking_lot to 0.8 (#1078) 2019-06-21 10:42:09 -04:00
Eliza Weisman d4adeeef2f trace: Remove the AsId trait (#1145)
While we're making breaking changes to `tokio-trace`, it would be good
to get rid of the `AsId` trait. The goal of span functions that are
generic over `Span`/`Id` can be achieved without the unnecessary
complexity of defining a new trait. This would also make the API added
to `tokio_trace_core::Event` in #1109 more consistent with the
`tokio-trace::Span` API.

This branch removes `AsId` from `tokio-trace` and replaces its uses with
`impl Into<Option<Id>>` and `impl Into<Option<&'a Id>>`. While `AsRef`
might be more semantically correct for the borrowed-`Id` conversion, its
signature doesn't permit conversion into an `Option`. Implementations of
`Into<Option<Id>>` and `Into<Option<&'a Id>>` have been added for 
`tokio_trace::Span`.

This is _technically_ a breaking API change, as it changes function
signatures. However, the existing macro syntax still works as-is, and
the tests which pass `&Id`, `&Span`, and `&Option<Id>` to the span
macros all still compile after this change.

Closes #1143

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-13 12:53:08 -07:00
Steven Fackler 4f6395b31c Make threadpool::Runtime methods take &self (#1140)
The runtime is inherently multi-threaded, so it's going to have to deal
with synchronization when submitting new tasks anyway. This allows a
runtime to be shared by multiple threads more easily when e.g. building
a blocking facade over a tokio-based API.
2019-06-10 12:54:27 -07:00
yanjhk 5c0b56278b Use ThreadPool's impl of spawn (#1139) 2019-06-10 11:23:12 -07:00
Eliza Weisman 41ca9a43de trace: Add shorthand syntax for local fields (#1103)
## Motivation

A common pattern in `tokio-trace` is to use the value of a local
variable as a field on a span or event. Currently, this requires code
like:
```rust
info!(foo = foo);
```
which is not particularly ergonomic given how commonly this occurs.
Struct initializers support a shorthand syntax for fields where the name
of the field is the same as a local variable, and `tokio-trace` should
as well.

## Solution

This branch adds support for syntax like
```rust
let foo = ...;
info!(foo);
```
and 
```rust
let foo = Foo {
    bar: ...,
    ...
};
info!(foo.bar)
```
to the `tokio-trace` span and event macros. This syntax also works with
the `Debug` and `Display` field shorthand.

The span macros previously used a field name with no value to indicate 
an uninitialized field. A new issue, #1138, has been opened for finding a
replacement syntax for uninitialized fields. Until then, the `tokio-trace` 
macros will no longer provide a way to create fields without values, 
although the `-core` API will continue to support this.

Closes #1062 

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-09 13:16:35 -07:00
Carl Lerche 8d0f102de8 Merge branch 'v0.1.x' into merge-0.1 2019-06-05 12:28:39 -07:00
Kevin Leimkuhler 5dcb379f6d Bump tokio-sync to 0.1.6 (#1123) 2019-06-05 12:19:06 -07:00
Kevin Leimkuhler 970f75f830 sync: Add Sync impl for Lock (#1117) 2019-06-04 17:04:35 -07:00
Kevin Leimkuhler 619efed28b sync: Add Sync impl for Lock (#1116)
Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-06-03 11:12:28 -07:00
Carl Lerche 18ed0be851 executor: remove unnecessary APIs from Enter. (#1115) 2019-05-31 11:11:10 -07:00
Carl Lerche 01052f930a Bump tokio version to v0.1.21. (#1113) 2019-05-30 14:39:30 -07:00
Lucio Franco 940f2c3431 Update tokio-trace-core to 0.2 (#1111)
Also includes 1b498e8aa2
2019-05-30 11:33:55 -07:00
Eliza Weisman 84d5a7f5a0 trace: Change Span::enter to return a guard, add Span::in_scope (#1076)
## Motivation

Currently, the primary way to use a span is to use `.enter` and pass a
closure to be executed under the span. While that is convenient in many
settings, it also comes with two decently inconvenient drawbacks:

 - It breaks control flow statements like `return`, `?`, `break`, and
   `continue`
 - It require re-indenting a potentially large chunk of code if you wish
   it to appear under a span

## Solution

This branch changes the `Span::enter` function to return a scope guard 
that exits the span when dropped, as in:
```rust
let guard = span.enter();

// code here is within the span

drop(guard);

// code here is no longer within the span
```
The method previously called `enter`, which takes a closure and 
executes it in the span's context, is now called `Span::in_scope`, and
was reimplemented on top of the new `enter` method. 

This is a breaking change to `tokio-trace` that will be part of the
upcoming 0.2 release.

Closes #1075 

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-24 15:24:13 -07:00
Carl Lerche 1b498e8aa2 Fix TCP poll_hup test (#1106)
This updates tests to track a fix applied in Mio. Previously, Mio
incorrectly fired HUP events. This was due to Mio mapping `RDHUP` to
HUP. The test is updated to correctly generate a HUP event.

Additionally, HUP events will be removed from all platforms except for
Linux. This is caused by the inability to reliably map kqueue events to
the epoll HUP behavior.
2019-05-24 14:08:07 -07:00
Eliza Weisman b2c53987d9 trace: Add shorthand for field::display and field::debug (#1088)
## Motivation

In `tokio-trace`, field values may be recorded as either a subset of
Rust primitive types or as `fmt::Display` and `fmt::Debug`
implementations. Currently, `tokio-trace` provides the `field::display`
and `field::debug` functions which wrap a type with a type that
implements `Value` using the wrapped type's `fmt::Display` or
`fmt::Debug` implementation. However, importing and using these
functions adds unnecessary boilerplate. 

In #1081, @jonhoo suggested adding shorthand syntax to the macros,
similar to that used by the `slog` crate, as a solution for the
wordiness of the current API.

## Solution

This branch adds `?` and `%` sigils to field values in the span and
event macros, which expand to the `field::debug` and `field::display`
wrappers, respectively. The shorthand sigils may be used in any position
where the macros take a field value.

For example:
```rust
trace_span!("foo", my_field = ?something, ...); // shorthand for `debug`
info!(foo = %value, bar = false, ...) // shorthand for `display`
```

Adding this shorthand required a fairly large change to how field
key-value pairs are handled by the macros --- since `%foo` and `%foo`
are not valid Rust expressions, we can no longer match repeated 
`$ident = $expr` patterns, and must now match field lists as repeated
token trees. The inner helper macros for constructing `FieldSet`s and
`ValueSet`s have to parse the token trees recursively. This added a
decent chunk of complexity, but fortunately we have a large number of
compile tests for the macros and I'm quite confident that all existing
invocations will still work.

Closes #1081

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-21 10:31:48 -07:00
Carl Lerche 38092010c4 Merge branch 'v0.1.x' 2019-05-14 11:50:44 -07:00
Carl Lerche 475dabe96d Release tokio v0.1.20, tokio-timer v0.2.21, and remove async-await-preview feature. (#1089)
The `async-await-preview` feature is removed as 0.1 will no longer track
Rust nightly.

This also bumps:
- tokio-timer (0.2.11).
2019-05-14 11:21:24 -07:00
Carl Lerche cb4aea394e Update Tokio to Rust 2018 (#1082) 2019-05-14 10:27:36 -07:00
Jeehoon Kang 79d8820050 Fix link in tokio-futures/README.md (#1085)
`tokio-futures/README.md`'s link to the examples was wrong.
2019-05-10 10:19:32 -07:00
Carl Lerche 951f2fd910 test: re-export macro dependencies (#1077)
Callers may not always have `futures` available at the root of the
crate. Re-exporting dependencies makes them available to the macro at a
deterministic location.
2019-05-03 20:43:40 -07:00
Carl Lerche 4ef736b9d5 async-await: add current_thread::Runtime::block_on_async (#1072)
This function is used by the Tokio macros introduced by #1058  but was
omitted from the PR.
2019-04-30 19:55:22 -07:00
Steven Fackler 219f24cbf1 timer: Replace Handle::deadline with Handle::timeout (#1074)
Deadline was deprecated a while ago and replaced with Timeout, but the
methods on Handle got missed.

Fixes #1071
2019-04-30 10:29:54 -07:00
Carl Lerche ea282efb2e ci: fix isRelease condition (#1066) 2019-04-29 10:37:00 -07:00
Michal 'vorner' Vaner 042224d33c signal: Smaller dependency (#1069)
The signal-hook library got split into lower-level and higher-level
parts. The tokio-signal uses only API from the lower-level one, so it
can depend on it directly.

The only effect of this change is smaller amount of compiled (and
unused) code during compilation. There's no change in the code actually
used.
2019-04-28 19:12:40 -07:00
Ian Hamlin 927eb80ad4 Fix an error in the mit-url in the README.md (#1068) 2019-04-27 12:56:40 -07:00
Carl Lerche 6a8934e897 Fix threadpool dependency (#1061) 2019-04-25 22:23:24 -04:00
Carl Lerche 0e400af78c Async/await polish (#1058)
A general refresh of Tokio's experimental async / await support.
2019-04-25 22:22:32 -04:00
Carl Lerche df702130d6 tcp: fix some tests that spuriously fail (#1060)
This does not remove all cases of using a fixed port in doc tests, but
removing some should reduce the likelihood of spurious failures.
2019-04-25 12:01:39 -07:00
Carl Lerche 949adbb887 chore: remember to remove path deps on release (#1057) 2019-04-24 10:42:39 -07:00
Ryan Dahl b2b796a228 rt: forward panic_handler to tokio::runtime::Builder (#1055) 2019-04-24 10:41:42 -07:00
Igor Gnatenko abb014efc2 tokio: Bump min version of tokio-sync (#1054)
It is needed for lock functionality which tokio now uses.
2019-04-24 08:24:59 -07:00
Lucio Franco e5cf0cc717 Introduce tokio-test crate (#1030) 2019-04-23 20:17:57 -07:00
Carl Lerche 62f34e15ce Bump tokio to 0.1.19. (#1053)
This also bumps:

- tokio-async-await (0.1.7)
- tokio-buf (0.1.1)
- tokio-sync (0.1.5)
- tokio-threadpool (0.1.14)
2019-04-22 15:12:25 -07:00
Eliza Weisman 3ebca76a9a trace: prepare tokio-trace for release (#1051) 2019-04-22 14:15:07 -07:00
Ryan Dahl fea1f780bc threadpool: add panic_handler (#1052) 2019-04-21 16:26:09 -07:00
Eliza Weisman 712ca84033 trace-core: prepare for 0.2 release (#1047) 2019-04-21 10:20:28 -07:00
Jon Gjengset cf06621998 tokio-sync: Add async mutual exclusion primitive (#964)
This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.

This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
2019-04-18 13:16:26 -04:00
Lucio Franco 7e51ab05e9 buf: Add IntoStream (#1048)
* buf: Add IntoStream

* Add debug implementation for IntoStream

* Add get_ref, get_mut and into_inner
2019-04-18 11:42:34 -04:00
Eliza Weisman 4bfa4ffcdf trace-core: Dispatchers unset themselves (#1033)
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans. 

Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.

Before:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          51 ns/iter (+/- 12)
test span_repeatedly         ... bench:       5,073 ns/iter (+/- 1,528)
test span_with_fields        ... bench:          56 ns/iter (+/- 49)
test span_with_fields_record ... bench:         363 ns/iter (+/- 61)
```

After:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          35 ns/iter (+/- 12)
test span_repeatedly         ... bench:       4,165 ns/iter (+/- 298)
test span_with_fields        ... bench:          48 ns/iter (+/- 12)
test span_with_fields_record ... bench:         363 ns/iter (+/- 91)
```

Closes #1032 

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-16 15:51:45 -07:00
Taiki Endo 88b942652c async-await: fix examples (#1050)
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
  on Rust 2018
* Fix an example in the documentation
2019-04-16 14:19:38 -04:00
Taiki Endo 5029e80a89 async-await: update to new futures_api (#1049) 2019-04-16 14:07:03 -04:00
Eliza Weismanandcsmoe 847fb59b17 trace-core: Introduce callsite classification in metadata (#1046)
## Motivation

To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.

## Solution

A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.

`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.

New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.

Fixes: #986
Closes: #1008

Co-Authored-By: csmoe <[email protected]>
2019-04-11 11:56:42 -07:00
Jane Lusby b4fe517a16 trace-core: add a function to rebuild cached interest (#1039)
## Motivation

Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.

However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`.  Even when filters change very
infrequently, they must still always be re-evaluated every time.

In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.

## Solution

This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.

## Breaking Change

In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.

Closes #1038

Co-Authored-By: yaahallo <[email protected]>
2019-04-10 13:51:05 -07:00
Simon Wollwage 7ae010f0f3 async-await: Use Context instead of Waker in poll (#1041)
Rust nightly std::future::Future recently changed Waker
to Context.

Change to use Context

Co-Authored-By: Kintaro <[email protected]>
2019-04-10 09:14:31 -07:00
Carl Lerche 9144b2ff53 sync: remove unnecessary imports (#1043) 2019-04-09 12:26:11 -07:00
Lev Eniseev 2c4549a18a Add example of blocking environment (#1036) 2019-04-09 12:10:15 -07:00
João Oliveira 4f819b7cd1 trace: fix counters example, Span IDs must be greater than zero (#1037)
## Motivation
tokio-trace counter example was panicking due to returning 0 as the first Span Id

## Solution
start ID's with 1 instead
2019-04-05 13:51:20 -07:00
Eliza Weisman 197f88f3bc trace: Change Span::enter and record to take &self (#1029)
## Motivation

The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.

Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
    span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
    span.record("bar" &false);
});
```

Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.

The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.

## Solution

This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows. 

In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`. 

## Notes

In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.

Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.

Before:
```
test enter_span              ... bench:          13 ns/iter (+/- 1)
```

...and after:
```
test enter_span              ... bench:           3 ns/iter (+/- 1)
```

Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.

I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 15:06:47 -07:00
Eliza Weisman 44f65afcc6 trace: Allow field names to be separated by .s (#1027)
## Motivation

In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.

## Solution

This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.

## Notes

The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.

Closes #1018.
Closes #1022.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 14:44:11 -07:00
Eliza Weisman 4271a9cd8d trace: Fix subscriber benchmarks panicking (#1028)
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-02 13:42:31 -07:00
Matthias Prechtl 9d8096b911 Improve documentation of Subscriber::record and Subscriber::event (#1026) 2019-04-02 12:44:01 -07:00
João Oliveira 597f271c08 trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation 

Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952). 

Closes #1013

## Solution 

Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels

## Notes 

I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
2019-04-02 11:29:23 -07:00
Taiki Endo 599955f716 Replace try! macro with ? operator (#1024) 2019-04-01 13:45:59 -07:00
Ivan Petkov 91bb0f73f5 signal: refactor Windows registrations to be lazy (#1001)
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
  so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
  when calling `global_init`

Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS

Fixes #999
2019-04-01 12:46:22 -07:00
Eliza Weisman 6c9d8abba9 trace: Make Span API functions taking IDs a little more flexible (#1021)
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.

Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-01 11:42:29 -07:00
Carl Lerche 824b7b6759 buf: stream and iter helpers (#1011) 2019-03-29 12:26:13 -07:00
Carl Lerche cb91dd274a buf: impl Error for CollectVecError (#1010) 2019-03-29 08:49:08 -07:00
Eliza Weisman ea7178b8c6 trace-core: Add slightly more useful debug impls (#1014)
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.

When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.

The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-28 14:47:06 -07:00
Carl Lerche a99b8e2e0b buf: impl size_hint for str types + reorg tests (#1012) 2019-03-28 14:02:45 -07:00
Carl Lerche 03859a7dcd buf: implement FromBufStream for Bytes (#1009) 2019-03-27 19:22:06 -07:00
Eliza Weisman d8177f81ac trace: Allow trace instrumentation to emit log records (#992)
## Motivation

`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.

Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.

## Solution

This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation. 

## Notes

In order to allow spans to log their names when they are entered and 
exited even when the span is disabled, this branch adds an 
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.

Fixes: #949

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-26 16:43:05 -07:00
Son ceca2a3cd6 chore: add license to tokio (#1006) 2019-03-26 08:41:41 -07:00
Son 1524ee4b60 trace: Add static level filtering (#987)
## Motivation

`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.

## Solution

We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.

Closes #959
2019-03-25 15:19:20 -07:00
MOZGIII 7793d63739 Corrected doc for tokio_buf SizeHint (#1003) 2019-03-25 16:54:49 -04:00
Red Hara e0e26bc223 Fix typo in README.md in examples (#1002) 2019-03-24 13:02:11 -04:00
Eliza Weisman 9c5cad037f trace-core: Add overrideable downcasting to Subscribers (#974)
## Motivation

In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to 
downcast the current `Dispatch` to a concrete subscriber type.

For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust

pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
    tokio_trace::dispatcher::with(|c| {
        if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
            s.do_elizatracing_correlation_magic(tt, et);
        }
    }); 
}
```

This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).

## Solution

This branch makes the following changes:
 * Add an object-safe `downcast_raw` method to the `Subscriber` trait,
   taking a `TypeId` and returning an `*const ()` if the type ID 
   matches the subscriber's type ID, or `None` if it does not, and
 * Add `is<T>` and `downcast_ref<T>` functions to `Subscriber` 
   and `Dispatch`, using `downcast_raw`.

Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_ 
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour 
somehow, can downcast to the inner type if they choose to.

[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204

Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 16:21:46 -07:00
Eliza Weisman 30330da11a chore: Fix examples not working with cargo run (#998)
* chore: Fix examples not working with `cargo run`

## Motivation

PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:

```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
error: no example target named `chat`

Did you mean `echo`?
```

## Solution

This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:

```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
   Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
   Compiling tokio-reactor v0.1.9
   Compiling tokio-threadpool v0.1.13
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.10
   Compiling tokio-uds v0.2.5
   Compiling tokio-udp v0.1.3
   Compiling tokio-tcp v0.1.3
   Compiling tokio-fs v0.1.6
   Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
    Finished dev [unoptimized + debuginfo] target(s) in 7.04s
     Running `target/debug/examples/chat`
server running on localhost:6142
```

Signed-off-by: Eliza Weisman <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 15:25:42 -07:00
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
557 changed files with 30484 additions and 28540 deletions
-22
View File
@@ -1,22 +0,0 @@
image: Visual Studio 2017
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- set RUST_BACKTRACE=1
- set LOOM_MAX_DURATION=10
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --no-fail-fast --target %TARGET%
+43
View File
@@ -0,0 +1,43 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_PREEMPTIONS: 2
RUSTFLAGS: -Dwarnings
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
# TODO: switch back to nightly
- sh rustup.sh -y --default-toolchain nightly-2019-08-21
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
test_script:
- . $HOME/.cargo/env
- cargo test --all
- cargo doc --all --no-deps
# TODO: Re-enable
# i686_test_script:
# - . $HOME/.cargo/env
# - |
# cargo test --all --exclude tokio-tls --exclude tokio-macros --target i686-unknown-freebsd
-131
View File
@@ -1,131 +0,0 @@
---
language: rust
sudo: false
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
- rust: stable
- rust: beta
- rust: nightly
env: ALLOW_FAILURES=true
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
#
# Tests are not run as tests may require newer versions of rust.
- rust: 1.26.0
script: |
cargo check --all
# Test combinations of enabled features.
- rust: stable
script: |
shopt -s expand_aliases
alias check="cargo check --no-default-features"
check
check --features codec
check --features fs
check --features io
check --features reactor
check --features rt-full
check --features tcp
check --features timer
check --features udp
check --features uds
# Test the async / await preview. We don't want to block PRs on this failing
# though.
- rust: nightly
env: ALLOW_FAILURES=true
script: |
cd tokio-async-await
cargo check --all
# This runs TSAN against nightly and allows failures to propagate up.
- rust: nightly-2018-11-18
env: TSAN=yes
script: |
set -e
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# This runs cargo +nightly doc
- name: nightly_docs
rust: nightly
env: ALLOW_FAILURES=true
script: cargo doc
allow_failures:
- rust: nightly
env: ALLOW_FAILURES=true
script: |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --exclude tokio-tls --target $TARGET
cargo check --tests --all --exclude tokio-tls --target $TARGET
else
# Limit the execution time of loom tests.
export LOOM_MAX_DURATION=10
cargo test --all --no-fail-fast
cargo doc --all
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = "linux" && $TARGET = ""
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+60 -4
View File
@@ -153,8 +153,6 @@ The type level example for `tokio_timer::Timeout` provides a good example of a
documentation test:
```
/// # extern crate futures;
/// # extern crate tokio;
/// // import the `timeout` function, usually this is done
/// // with `use tokio::prelude::*`
/// use tokio::prelude::FutureExt;
@@ -192,8 +190,6 @@ If this were a documentation test for the `Timeout::new` function, then the
example would explicitly use `Timeout::new`. For example:
```
/// # extern crate futures;
/// # extern crate tokio;
/// use tokio::timer::Timeout;
/// use futures::Future;
/// use futures::sync::oneshot;
@@ -385,3 +381,63 @@ _Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
## Releasing
Since the Tokio project consists of a number of crates, many of which depend on
each other, releasing new versions to crates.io can involve some complexities.
When releasing a new version of a crate, follow these steps:
1. **Ensure that the release crate has no path dependencies.** When the HEAD
version of a Tokio crate requires unreleased changes in another Tokio crate,
the crates.io dependency on the second crate will be replaced with a path
dependency. Crates with path dependencies cannot be published, so before
publishing the dependent crate, any path dependencies must also be published.
This should be done through a form of depth-first tree traversal:
1. Starting with the first path dependency in the crate to be released,
inspect the `Cargo.toml` for the dependency. If the dependency has any
path dependencies of its own, repeat this step with the first such
dependency.
2. Begin the release process for the path dependency.
3. Once the path dependency has been published to crates.io, update the
dependent crate to depend on the crates.io version.
4. When all path dependencies have been published, the dependent crate may
be published.
To verify that a crate is ready to publish, run:
```bash
bin/publish --dry-run <CRATE NAME> <CRATE VERSION>
```
2. **Update Cargo metadata.** After releasing any path dependencies, update the
`version` field in `Cargo.toml` to the new version, and the `documentation`
field to the docs.rs URL of the new version.
3. **Update other documentation links.** Update the `#![doc(html_root_url)]`
attribute in the crate's `lib.rs` and the "Documentation" link in the crate's
`README.md` to point to the docs.rs URL of the new version.
4. **Update the changelog for the crate.** Each crate in the Tokio repository
has its own `CHANGELOG.md` in that crate's subdirectory. Any changes to that
crate since the last release should be added to the changelog. Change
descriptions may be taken from the Git history, but should be edited to
ensure a consistent format, based on [Keep A Changelog][keep-a-changelog].
Other entries in that crate's changelog may also be used for reference.
5. **Perform a final audit for breaking changes.** Compare the HEAD version of
crate with the Git tag for the most recent release version. If there are any
breaking API changes, determine if those changes can be made without breaking
existing APIs. If so, resolve those issues. Otherwise, if it is necessary to
make a breaking release, update the version numbers to reflect this.
6. **Open a pull request with your changes.** Once that pull request has been
approved by a maintainer and the pull request has been merged, continue to
the next step.
7. **Release the crate.** Run the following command:
```bash
bin/publish <NAME OF CRATE> <VERSION>
```
Your editor and prompt you to edit a message for the tag. Copy the changelog
entry for that release version into your editor and close the window.
[keep-a-changelog]: https://github.com/olivierlacan/keep-a-changelog/blob/master/CHANGELOG.md
+5 -133
View File
@@ -1,144 +1,16 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.15"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.15/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio-async-await",
"tokio-buf",
"tokio",
"tokio-codec",
"tokio-current-thread",
"tokio-executor",
"tokio-fs",
"tokio-io",
"tokio-reactor",
"tokio-signal",
"tokio-macros",
"tokio-net",
"tokio-sync",
"tokio-threadpool",
"tokio-test",
"tokio-timer",
"tokio-tcp",
"tokio-tls",
"tokio-udp",
"tokio-uds",
"build-tests",
]
[features]
default = [
"codec",
"fs",
"io",
"reactor",
"rt-full",
"sync",
"tcp",
"timer",
"udp",
"uds",
]
codec = ["tokio-codec"]
fs = ["tokio-fs"]
io = ["bytes", "tokio-io"]
reactor = ["io", "mio", "tokio-reactor"]
rt-full = [
"num_cpus",
"reactor",
"timer",
"tokio-current-thread",
"tokio-executor",
"tokio-threadpool",
]
sync = ["tokio-sync"]
tcp = ["tokio-tcp"]
timer = ["tokio-timer"]
udp = ["tokio-udp"]
uds = ["tokio-uds"]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
# Only non-optional dependency...
futures = "0.1.20"
# Everything else is optional...
bytes = { version = "0.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
tokio-codec = { version = "0.1.0", path = "tokio-codec", optional = true }
tokio-current-thread = { version = "0.1.3", path = "tokio-current-thread", optional = true }
tokio-fs = { version = "0.1.3", path = "tokio-fs", optional = true }
tokio-io = { version = "0.1.6", path = "tokio-io", optional = true }
tokio-executor = { version = "0.1.5", path = "tokio-executor", optional = true }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor", optional = true }
tokio-sync = { version = "0.1.0", path = "tokio-sync", optional = true }
tokio-threadpool = { version = "0.1.8", path = "tokio-threadpool", optional = true }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp", optional = true }
tokio-udp = { version = "0.1.0", path = "tokio-udp", optional = true }
tokio-timer = { version = "0.2.8", path = "tokio-timer", optional = true }
# Needed until `reactor` is removed from `tokio`.
mio = { version = "0.6.14", optional = true }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds", optional = true }
[dev-dependencies]
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
#tokio = { path = "." }
#tokio-async-await = { path = "./tokio-async-await" }
#tokio-codec = { path = "./tokio-codec" }
#tokio-current-thread = { path = "./tokio-current-thread" }
#tokio-executor = { path = "./tokio-executor" }
#tokio-fs = { path = "./tokio-fs" }
#tokio-io = { path = "./tokio-io" }
#tokio-reactor = { path = "./tokio-reactor" }
#tokio-signal = { path = "./tokio-signal" }
#tokio-tcp = { path = "./tokio-tcp" }
#tokio-threadpool = { path = "./tokio-threadpool" }
#tokio-timer = { path = "./tokio-timer" }
#tokio-tls = { path = "./tokio-tls" }
#tokio-udp = { path = "./tokio-udp" }
#tokio-uds = { path = "./tokio-uds" }
+73 -73
View File
@@ -1,5 +1,7 @@
# Tokio
**NOTE**: Tokio's [`master`](https://github.com/tokio-rs/tokio) is currently undergoing heavy development. This branch and the alpha releases will see API breaking changes and there are currently significant performance regressions that still need to be fixed before the final release. Use the [`v0.1.x`](https://github.com/tokio-rs/tokio/tree/v0.1.x) branch for stable releases.
A runtime for writing reliable, asynchronous, and slim applications with
the Rust programming language. It is:
@@ -14,30 +16,23 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[mit-url]: LICENSE
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio/0.1.15/tokio) |
[Guides](https://tokio.rs/docs/) |
[API Docs](https://docs.rs/tokio/latest/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
## Overview
Tokio is an event-driven, non-blocking I/O platform for writing
@@ -45,63 +40,65 @@ asynchronous applications with the Rust programming language. At a high
level, it provides a few major components:
* A multithreaded, work-stealing based task [scheduler].
* A [reactor] backed by the operating system's event queue (epoll, kqueue,
* A reactor backed by the operating system's event queue (epoll, kqueue,
IOCP, etc...).
* Asynchronous [TCP and UDP][net] sockets.
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/latest/tokio/net/index.html
[scheduler]: https://docs.rs/tokio/latest/tokio/runtime/index.html
## Example
A basic TCP echo server with Tokio:
```rust
extern crate tokio;
use tokio::prelude::*;
use tokio::io::copy;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::net::SocketAddr;
fn main() {
// Bind the server's socket.
let addr = "127.0.0.1:12345".parse().unwrap();
let listener = TcpListener::bind(&addr)
.expect("unable to bind TCP listener");
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "127.0.0.1:8080".parse::<SocketAddr>()?;
let mut listener = TcpListener::bind(&addr).await?;
// Pull out a stream of sockets for incoming connections
let server = listener.incoming()
.map_err(|e| eprintln!("accept failed = {:?}", e))
.for_each(|sock| {
// Split up the reading and writing parts of the
// socket.
let (reader, writer) = sock.split();
loop {
let (mut socket, _) = listener.accept().await?;
// A future that echos the data and returns how
// many bytes were copied...
let bytes_copied = copy(reader, writer);
tokio::spawn(async move {
let mut buf = [0; 1024];
// ... after which we'll print what happened.
let handle_conn = bytes_copied.map(|amt| {
println!("wrote {:?} bytes", amt)
}).map_err(|err| {
eprintln!("IO error {:?}", err)
});
// In a loop, read data from the socket and write the data back.
loop {
let n = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => return,
Ok(n) => n,
Err(e) => {
println!("failed to read from socket; err = {:?}", e);
return;
}
};
// Spawn the future as a concurrent task.
tokio::spawn(handle_conn)
// Write the data back
if let Err(e) = socket.write_all(&buf[0..n]).await {
println!("failed to write to socket; err = {:?}", e);
return;
}
}
});
// Start the Tokio runtime
tokio::run(server);
}
}
```
More examples can be found [here](examples).
More examples can be found [here](tokio/examples). Note that the `master` branch
is currently being updated to use `async` / `await`. The examples are
not fully ported. Examples for stable Tokio can be found
[here](https://github.com/tokio-rs/tokio/tree/v0.1.x/tokio/examples).
## Getting Help
@@ -110,6 +107,8 @@ First, see if the answer to your question can be found in the [Guides] or the
the [Tokio Gitter channel][chat]. We would be happy to try to answer your
question. Last, if that doesn't work, try opening an [issue] with the question.
[Guides]: https://tokio.rs/docs/
[API documentation]: https://docs.rs/tokio/latest/tokio
[chat]: https://gitter.im/tokio-rs/tokio
[issue]: https://github.com/tokio-rs/tokio/issues/new
@@ -129,46 +128,47 @@ have greater guarantees of stability.
The crates included as part of Tokio are:
* [`tokio-async-await`]: Experimental `async` / `await` support.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-current-thread`]: Schedule the execution of futures on the current
thread.
* [`tokio-executor`]: Task execution related traits and utilities.
* [`tokio-executor`]: Task executors and related utilities. Includes a
single-threaded executor and a multi-threaded, work-stealing, executor.
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
* [`tokio-macros`]: Macros for usage with Tokio.
* [`tokio-tcp`]: TCP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-threadpool`]: Schedules the execution of futures across a pool of
threads.
* [`tokio-net`]: Event loop that drives I/O resources as well as TCP, UDP, and
unix domain socket apis.
* [ `tokio-timer`]: Time related APIs.
* [`tokio-udp`]: UDP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-uds`]: Unix Domain Socket bindings for use with `tokio-io` and
`tokio-reactor`.
[`tokio-async-await`]: tokio-async-await
[`tokio-codec`]: tokio-codec
[`tokio-current-thread`]: tokio-current-thread
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-io`]: tokio-io
[`tokio-reactor`]: tokio-reactor
[`tokio-tcp`]: tokio-tcp
[`tokio-threadpool`]: tokio-threadpool
[`tokio-macros`]: tokio-macros
[`tokio-net`]: tokio-net
[`tokio-timer`]: tokio-timer
[`tokio-udp`]: tokio-udp
[`tokio-uds`]: tokio-uds
## Related Projects
In addition to the crates in this repository, the Tokio project also maintains
several other libraries, including:
* [`tracing`] (formerly `tokio-trace`): A framework for application-level
tracing and async-aware diagnostics.
* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers
`tokio`.
* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.
[`tracing`]: https://github.com/tokio-rs/tracing
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Supported Rust Versions
+131
View File
@@ -0,0 +1,131 @@
trigger: ["master", "std-future"]
pr: ["master", "std-future"]
variables:
RUSTFLAGS: -Dwarnings
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
rust: beta
name: rustfmt
# Apply clippy lints to all crates
- template: ci/azure-clippy.yml
parameters:
rust: beta
name: clippy
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
rust: beta
displayName: Test tokio
cross: true
crates:
tokio:
- codec
- fs
- io
- rt-full
- net
- sync
- tcp
- timer
- udp
- uds
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates (cross) -
cross: true
rust: beta
crates:
tokio-fs: []
tokio-net:
- process
- signal
- tcp
- udp
- uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
rust: beta
crates:
tokio-codec: []
tokio-executor:
- current-thread
- threadpool
tokio-io:
- util
tokio-sync:
- async-traits
tokio-macros: []
tokio-timer:
- async-traits
tokio-test: []
# Test compilation failure
- template: ci/azure-test-stable.yml
parameters:
name: test_features
displayName: Test feature flags
rust: beta
crates:
build-tests:
- tokio-executor
- tokio-net
- executor-without-current-thread
- macros-invalid-input
- net-no-features
- net-with-tcp
- net-with-udp
- net-with-uds
- tokio-no-features
- tokio-with-net
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross
rust: beta
# # This represents the minimum Rust version supported by
# # Tokio. Updating this should be done in a dedicated PR and
# # cannot be greater than two 0.x releases prior to the
# # current stable.
# #
# # Tests are not run as tests may require newer versions of
# # rust.
# - template: ci/azure-check-minrust.yml
# parameters:
# name: minrust
# rust_version: 1.34.0
#
# - template: ci/azure-tsan.yml
# parameters:
# name: tsan
# rust: beta
- template: ci/azure-deploy-docs.yml
parameters:
rust: beta
dependsOn:
- rustfmt
- clippy
- test_tokio
- test_sub_cross
- test_linux
- test_features
# - test_nightly
- cross
# - minrust
# - tsan
Executable
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
set -e
USAGE="Publish a new release of a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-v, --verbose Use verbose Cargo output
-d, --dry-run Perform a dry run (do not publish or tag the release)
-h, --help Show this help text and exit"
DRY_RUN=""
VERBOSE=""
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
verify() {
status "Verifying" "if $CRATE v$VERSION can be released"
ACTUAL=$(cargo pkgid | sed -n 's/.*#\(.*\)/\1/p')
if [ "$ACTUAL" != "$VERSION" ]; then
err "expected to release version $VERSION, but Cargo.toml contained $ACTUAL"
exit 1
fi
if git tag -l | grep -Fxq "$TAG" ; then
err "git tag \`$TAG\` already exists"
exit 1
fi
PATH_DEPS=$(grep -F "path = \"" Cargo.toml | sed -e 's/^/ /')
if [ -n "$PATH_DEPS" ]; then
err "crate \`$CRATE\` contained path dependencies:\n$PATH_DEPS"
echo "path dependencies must be removed prior to release"
exit 1
fi
}
release() {
status "Releasing" "$CRATE v$VERSION"
cargo package $VERBOSE
cargo publish $VERBOSE $DRY_RUN
status "Tagging" "$TAG"
if [ -n "$DRY_RUN" ]; then
echo "# git tag $TAG && git push --tags"
else
git tag "$TAG" && git push --tags
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-v|--verbose)
VERBOSE="--verbose"
set +x
shift
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
(cd "$CRATE" && verify && release )
else
err "no such crate \"$CRATE\""
exit 1
fi
Executable
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
set -e
USAGE="Update links to docs.rs in a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-d, --dry-run Perform a dry run (do not modify any file)
-h, --help Show this help text and exit"
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
c1grep() { grep "$@" || test $? = 1; }
update_versions_in_doc() {
# Print what is being/would be done
if [ -n "$DRY_RUN" ]; then
local MSG="Would change:"
else
local MSG="Updating:"
fi
git grep -lr "docs.rs/$CRATE/" \
| xargs sed --quiet \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|gp" \
| sed -e "s/^/$MSG /"
# Apply changes if not in dry run
if [ -z "$DRY_RUN" ]; then
git grep -lr "docs.rs/$CRATE/" \
| xargs sed -i \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|g"
fi
}
update() {
update_versions_in_doc
}
show_outdated() {
OUTDATED=$(git grep -rn "docs.rs/$CRATE/" \
| c1grep -v "$VERSION" \
| sed -e 's/^/ - /')
if [[ -n "$OUTDATED" ]]; then
echo "Found the following links to docs.rs with an outdated version:"
echo "$OUTDATED"
echo
else
echo "Nothing to do."
exit 1
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
# Does not cd in order to update everywhere
show_outdated && update
else
err "no such crate \"$CRATE\""
exit 1
fi
+27
View File
@@ -0,0 +1,27 @@
[package]
name = "build-tests"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[features]
executor-without-current-thread = ["tokio-executor"]
macros-invalid-input = ["tokio/rt-full"]
net-no-features = ["tokio-net"]
net-with-tcp = ["tokio-net/tcp"]
net-with-udp = ["tokio-net/udp"]
net-with-uds = ["tokio-net/uds"]
net-with-process = ["tokio-net/process"]
tokio-no-features = ["tokio"]
tokio-with-net = ["tokio/net"]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
tokio-executor = { path = "../tokio-executor", optional = true }
tokio-net = { path = "../tokio-net", optional = true }
tokio = { path = "../tokio", optional = true, default-features = false }
[dev-dependencies]
trybuild = "1.0"
+2
View File
@@ -0,0 +1,2 @@
Tests the various combination of feature flags. This is broken out to a separate
crate to work around limitations with cargo features.
+8
View File
@@ -0,0 +1,8 @@
#[cfg(feature = "tokio-executor")]
pub use tokio_executor;
#[cfg(feature = "tokio-net")]
pub use tokio_net;
#[cfg(feature = "tokio")]
pub use tokio;
@@ -0,0 +1,3 @@
use build_tests::tokio_executor::current_thread;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_executor::current_thread`
--> $DIR/executor_without_current_thread.rs:1:5
|
1 | use build_tests::tokio_executor::current_thread;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `current_thread` in `tokio_executor`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,28 @@
use build_tests::tokio;
#[tokio::main]
fn main_is_not_async() {}
#[tokio::main]
async fn main_fn_has_args(_x: u8) {}
#[tokio::main(foo)]
async fn main_attr_has_unknown_args() {}
#[tokio::main(multi_thread::bar)]
async fn main_attr_has_path_args() {}
#[tokio::test]
fn test_is_not_async() {}
#[tokio::test]
async fn test_fn_has_args(_x: u8) {}
#[tokio::test(foo)]
async fn test_attr_has_args() {}
#[tokio::test]
#[test]
async fn test_has_second_test_attr() {}
fn main() {}
@@ -0,0 +1,47 @@
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:4:1
|
4 | fn main_is_not_async() {}
| ^^
error: the main function cannot accept arguments
--> $DIR/macros_invalid_input.rs:7:27
|
7 | async fn main_fn_has_args(_x: u8) {}
| ^^^^^^
error: Unknown attribute foo is specified
--> $DIR/macros_invalid_input.rs:9:15
|
9 | #[tokio::main(foo)]
| ^^^
error: Must have specified ident
--> $DIR/macros_invalid_input.rs:12:15
|
12 | #[tokio::main(multi_thread::bar)]
| ^^^^^^^^^^^^^^^^^
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:16:1
|
16 | fn test_is_not_async() {}
| ^^
error: the test function cannot accept arguments
--> $DIR/macros_invalid_input.rs:19:27
|
19 | async fn test_fn_has_args(_x: u8) {}
| ^^^^^^
error: unexpected token
--> $DIR/macros_invalid_input.rs:21:15
|
21 | #[tokio::test(foo)]
| ^^^
error: second test attribute is supplied
--> $DIR/macros_invalid_input.rs:25:1
|
25 | #[test]
| ^^^^^^^
@@ -0,0 +1,4 @@
use build_tests::tokio_net::tcp;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_net::tcp`
--> $DIR/net_without_tcp_missing_tcp.rs:1:5
|
1 | use build_tests::tokio_net::tcp;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `tcp` in `tokio_net`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,4 @@
use build_tests::tokio_net::udp;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_net::udp`
--> $DIR/net_without_udp_missing_udp.rs:1:5
|
1 | use build_tests::tokio_net::udp;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `udp` in `tokio_net`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,4 @@
use build_tests::tokio_net::uds;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_net::uds`
--> $DIR/net_without_uds_missing_uds.rs:1:5
|
1 | use build_tests::tokio_net::uds;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `uds` in `tokio_net`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,3 @@
use build_tests::tokio::net;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio::net`
--> $DIR/tokio_without_net_missing_net.rs:1:5
|
1 | use build_tests::tokio::net;
| ^^^^^^^^^^^^^^^^^^^^^^^ no `net` in `tokio`
For more information about this error, try `rustc --explain E0432`.
+62
View File
@@ -0,0 +1,62 @@
#![allow(unused_imports)]
#[test]
#[cfg(feature = "tokio-net")]
fn net_default() {
use build_tests::tokio_net::driver::{set_default, Handle, Reactor, Registration};
use build_tests::tokio_net::util::PollEvented;
}
#[test]
#[cfg(feature = "net-with-tcp")]
fn net_with_tcp() {
use build_tests::tokio_net::tcp;
}
#[test]
#[cfg(feature = "net-with-udp")]
fn net_with_udp() {
use build_tests::tokio_net::udp;
}
#[test]
#[cfg(feature = "net-with-uds")]
fn net_with_uds() {
use build_tests::tokio_net::uds;
}
#[test]
#[cfg(feature = "net-with-process")]
fn net_with_process() {
use build_tests::tokio_net::process;
}
#[test]
#[cfg(feature = "tokio-with-net")]
fn tokio_with_net() {
// net is present
use build_tests::tokio::net;
}
#[test]
fn compile_fail() {
let t = trybuild::TestCases::new();
#[cfg(feature = "executor-without-current-thread")]
t.compile_fail("tests/fail/executor_without_current_thread.rs");
#[cfg(feature = "macros-invalid-input")]
t.compile_fail("tests/fail/macros_invalid_input.rs");
#[cfg(feature = "net-no-features")]
{
t.compile_fail("tests/fail/net_without_tcp_missing_tcp.rs");
t.compile_fail("tests/fail/net_without_udp_missing_udp.rs");
t.compile_fail("tests/fail/net_without_uds_missing_uds.rs");
}
#[cfg(feature = "tokio-no-features")]
t.compile_fail("tests/fail/tokio_without_net_missing_net.rs");
drop(t);
}
+29
View File
@@ -0,0 +1,29 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+16
View File
@@ -0,0 +1,16 @@
jobs:
- job: ${{ parameters.name }}
displayName: Clippy
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add clippy
cargo clippy --version
displayName: Install clippy
- script: |
cargo clippy --all --all-features -- -A clippy::mutex-atomic
displayName: cargo clippy --all
+44
View File
@@ -0,0 +1,44 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
i686:
vmImage: ubuntu-16.04
target: i686-unknown-linux-gnu
powerpc:
vmImage: ubuntu-16.04
target: powerpc-unknown-linux-gnu
powerpc64:
vmImage: ubuntu-16.04
target: powerpc64-unknown-linux-gnu
mips:
vmImage: ubuntu-16.04
target: mips-unknown-linux-gnu
arm:
vmImage: ubuntu-16.04
target: arm-unknown-linux-gnueabi
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: sudo apt-get update
displayName: apt-get update
- script: sudo apt-get install gcc-multilib
displayName: Install gcc-multilib
- script: cargo install cross
displayName: Install cross
# Always patch
- template: azure-patch-crates.yml
- script: cross check --all --exclude tokio-tls --target $(target)
displayName: Check source
# - script: cross check --tests --all --exclude tokio-tls --target $(target)
# displayName: Check tests
+39
View File
@@ -0,0 +1,39 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
# rust_version: stable
rust_version: ${{ parameters.rust }}
- script: |
cargo doc --all --no-deps --all-features
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+33
View File
@@ -0,0 +1,33 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain none
export PATH=$PATH:$HOME/.cargo/bin
rustup toolchain install $RUSTUP_TOOLCHAIN
rustup default $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain none
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
rustup toolchain install %RUSTUP_TOOLCHAIN%
rustup default %RUSTUP_TOOLCHAIN%
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustup toolchain list
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%B' | grep -qF '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+17
View File
@@ -0,0 +1,17 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add rustfmt
cargo fmt --version
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+19
View File
@@ -0,0 +1,19 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
# Check benches
- script: cargo check --benches --all
displayName: Check benchmarks
+61
View File
@@ -0,0 +1,61 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
# rust_version: stable
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
# Run with default crate features
- script: cargo test
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
# Run with each specified feature
- ${{ each feature in crate.value }}:
- script: cargo test --no-default-features --features ${{ feature }}
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test --features ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
# Run with default crate features
- script: cargo test
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
# Run with each specified feature
- ${{ each feature in crate.value }}:
- script: cargo test --no-default-features --features ${{ feature }}
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test --features ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-executor --tests --features threadpool
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+13
View File
@@ -0,0 +1,13 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-codec = { path = "tokio-codec" }
tokio-executor = { path = "tokio-executor" }
tokio-fs = { path = "tokio-fs" }
tokio-io = { path = "tokio-io" }
tokio-macros = { path = "tokio-macros" }
tokio-net = { path = "tokio-net" }
tokio-sync = { path = "tokio-sync" }
tokio-timer = { path = "tokio-timer" }
tokio-tls = { path = "tokio-tls" }
+2
View File
@@ -8,6 +8,8 @@ race:Weak*drop
# `std` mpsc is not used in any Tokio code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
race:std*lang_start
race:drop*std::thread*
# Probably more fences in std.
race:__call_tls_dtors
-60
View File
@@ -1,60 +0,0 @@
## Examples of how to use Tokio
This directory contains a number of examples showcasing various capabilities of
the `tokio` crate.
All examples can be executed with:
```
cargo run --example $name
```
A high level description of each example is:
* [`hello_world`](hello_world.rs) - a tiny server that writes "hello world" to
all connected clients and then terminates the connection, should help see how
to create and initialize `tokio`.
* [`echo`](echo.rs) - this is your standard TCP "echo server" which accepts
connections and then echos back any contents that are read from each connected
client.
* [`print_each_packet`](print_each_packet.rs) - this server will create a TCP
listener, accept connections in a loop, and put down in the stdout everything
that's read off of each TCP connection.
* [`echo-udp`](echo-udp.rs) - again your standard "echo server", except for UDP
instead of TCP. This will echo back any packets received to the original
sender.
* [`connect`](connect.rs) - this is a `nc`-like clone which can be used to
interact with most other examples. The program creates a TCP connection or UDP
socket to sends all information read on stdin to the remote peer, displaying
any data received on stdout. Often quite useful when interacting with the
various other servers here!
* [`chat`](chat.rs) - this spins up a local TCP server which will broadcast from
any connected client to all other connected clients. You can connect to this
in multiple terminals and use it to chat between the terminals.
* [`chat-combinator`](chat-combinator.rs) - Similar to `chat`, but this uses a
much more functional programming approach using combinators.
* [`proxy`](proxy.rs) - an example proxy server that will forward all connected
TCP clients to the remote address specified when starting the program.
* [`tinyhttp`](tinyhttp.rs) - a tiny HTTP/1.1 server which doesn't support HTTP
request bodies showcasing running on multiple cores, working with futures and
spawning tasks, and finally framing a TCP connection to discrete
request/response objects.
* [`tinydb`](tinydb.rs) - an in-memory database which shows sharing state
between all connected clients, notably the key/value store of this database.
* [`udp-client`](udp-client.rs) - a simple `send_dgram`/`recv_dgram` example.
* [`manual-runtime`](manual-runtime.rs) - manually composing a runtime.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
-167
View File
@@ -1,167 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! Because we are here running the reactor/executor on the same thread instead
//! of a threadpool, we can avoid full synchronization with Arc + Mutex and use
//! Rc + RefCell instead. The max performance is however limited to a CPU HW
//! thread.
//!
//! You can test this out by running:
//!
//! cargo run --example chat-combinator-current-thread
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
extern crate futures;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::runtime::current_thread::{Runtime, TaskExecutor};
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::rc::Rc;
use std::cell::RefCell;
fn main() -> Result<(), Box<std::error::Error>> {
let mut runtime = Runtime::new().unwrap();
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio current_thread runtime, so it will be single-
// threaded. The `Rc<RefCell<...>>` allows state to be shared across the tasks.
let connections = Rc::new(RefCell::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| {println!("failed to accept socket; error = {:?}", e); e})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
let mut conns = connections.borrow_mut();
conns.insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.borrow_mut();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn locally a task to process the connection
TaskExecutor::current().spawn_local(Box::new(connection.then(move |_| {
let mut conns = connections.borrow_mut();
conns.remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}))).unwrap();
Ok(())
})
.map_err(|err| println!("error occurred: {:?}", err));
// Spawn srv itself
runtime.spawn(srv);
// Execute server
runtime.run().unwrap();
Ok(())
}
-152
View File
@@ -1,152 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
extern crate futures;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::sync::{Arc, Mutex};
fn main() -> Result<(), Box<std::error::Error>> {
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio runtime, so it will be multi-threaded. The
// `Arc<Mutex<...>>` allows state to be shared across the threads.
let connections = Arc::new(Mutex::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| {println!("failed to accept socket; error = {:?}", e); e})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
connections.lock().unwrap().insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.lock().unwrap();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn a task to process the connection
tokio::spawn(connection.then(move |_| {
connections.lock().unwrap().remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}));
Ok(())
})
.map_err(|err| println!("error occurred: {:?}", err));
// execute server
tokio::run(srv);
Ok(())
}
-475
View File
@@ -1,475 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This example is explicitly more verbose than it has to be. This is to
//! illustrate more concepts.
//!
//! A chat server for telnet clients. After a telnet client connects, the first
//! line should contain the client's name. After that, all lines sent by a
//! client are broadcasted to all other connected clients.
//!
//! Because the client is telnet, lines are delimited by "\r\n".
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
//! You can run the `telnet` command in any number of additional windows.
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
#[macro_use]
extern crate futures;
extern crate bytes;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use std::collections::HashMap;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<Bytes>;
/// Shorthand for the receive half of the message channel.
type Rx = mpsc::UnboundedReceiver<Bytes>;
/// Data that is shared between all peers in the chat server.
///
/// This is the set of `Tx` handles for all connected clients. Whenever a
/// message is received from a client, it is broadcasted to all peers by
/// iterating over the `peers` entries and sending a copy of the message on each
/// `Tx`.
struct Shared {
peers: HashMap<SocketAddr, Tx>,
}
/// The state for each connected client.
struct Peer {
/// Name of the peer.
///
/// When a client connects, the first line sent is treated as the client's
/// name (like alice or bob). The name is used to preface all messages that
/// arrive from the client so that we can simulate a real chat server:
///
/// ```text
/// alice: Hello everyone.
/// bob: Welcome to telnet chat!
/// ```
name: BytesMut,
/// The TCP socket wrapped with the `Lines` codec, defined below.
///
/// This handles sending and receiving data on the socket. When using
/// `Lines`, we can work at the line level instead of having to manage the
/// raw byte operations.
lines: Lines,
/// Handle to the shared chat state.
///
/// This is used to broadcast messages read off the socket to all connected
/// peers.
state: Arc<Mutex<Shared>>,
/// Receive half of the message channel.
///
/// This is used to receive messages from peers. When a message is received
/// off of this `Rx`, it will be written to the socket.
rx: Rx,
/// Client socket address.
///
/// The socket address is used as the key in the `peers` HashMap. The
/// address is saved so that the `Peer` drop implementation can clean up its
/// entry.
addr: SocketAddr,
}
/// Line based codec
///
/// This decorates a socket and presents a line based read / write interface.
///
/// As a user of `Lines`, we can focus on working at the line level. So, we send
/// and receive values that represent entire lines. The `Lines` codec will
/// handle the encoding and decoding as well as reading from and writing to the
/// socket.
#[derive(Debug)]
struct Lines {
/// The TCP socket.
socket: TcpStream,
/// Buffer used when reading from the socket. Data is not returned from this
/// buffer until an entire line has been read.
rd: BytesMut,
/// Buffer used to stage data before writing it to the socket.
wr: BytesMut,
}
impl Shared {
/// Create a new, empty, instance of `Shared`.
fn new() -> Self {
Shared {
peers: HashMap::new(),
}
}
}
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
// Create a channel for this peer
let (tx, rx) = mpsc::unbounded();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
Peer {
name,
lines,
state,
rx,
addr,
}
}
}
/// This is where a connected client is managed.
///
/// A `Peer` is also a future representing completely processing the client.
///
/// When a `Peer` is created, the first line (representing the client's name)
/// has already been read. When the socket closes, the `Peer` future completes.
///
/// While processing, the peer future implementation will:
///
/// 1) Receive messages on its message channel and write them to the socket.
/// 2) Receive messages from the socket and broadcast them to all peers.
///
impl Future for Peer {
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<(), io::Error> {
// Tokio (and futures) use cooperative scheduling without any
// preemption. If a task never yields execution back to the executor,
// then other tasks may be starved.
//
// To deal with this, robust applications should not have any unbounded
// loops. In this example, we will read at most `LINES_PER_TICK` lines
// from the client on each tick.
//
// If the limit is hit, the current task is notified, informing the
// executor to schedule the task again asap.
const LINES_PER_TICK: usize = 10;
// Receive all messages from peers.
for i in 0..LINES_PER_TICK {
// Polling an `UnboundedReceiver` cannot fail, so `unwrap` here is
// safe.
match self.rx.poll().unwrap() {
Async::Ready(Some(v)) => {
// Buffer the line. Once all lines are buffered, they will
// be flushed to the socket (right below).
self.lines.buffer(&v);
// If this is the last iteration, the loop will break even
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
task::current().notify();
}
}
_ => break,
}
}
// Flush the write buffer to the socket
let _ = self.lines.poll_flush()?;
// Read new lines from the socket
while let Async::Ready(line) = self.lines.poll()? {
println!("Received line ({:?}) : {:?}", self.name, line);
if let Some(message) = line {
// Append the peer's name to the front of the line:
let mut line = self.name.clone();
line.extend_from_slice(b": ");
line.extend_from_slice(&message);
line.extend_from_slice(b"\r\n");
// We're using `Bytes`, which allows zero-copy clones (by
// storing the data in an Arc internally).
//
// However, before cloning, we must freeze the data. This
// converts it from mutable -> immutable, allowing zero copy
// cloning.
let line = line.freeze();
// Now, send the line to all other peers
for (addr, tx) in &self.state.lock().unwrap().peers {
// Don't send the message to ourselves
if *addr != self.addr {
// The send only fails if the rx half has been dropped,
// however this is impossible as the `tx` half will be
// removed from the map before the `rx` is dropped.
tx.unbounded_send(line.clone()).unwrap();
}
}
} else {
// EOF was reached. The remote client has disconnected. There is
// nothing more to do.
return Ok(Async::Ready(()));
}
}
// As always, it is important to not just return `NotReady` without
// ensuring an inner future also returned `NotReady`.
//
// We know we got a `NotReady` from either `self.rx` or `self.lines`, so
// the contract is respected.
Ok(Async::NotReady)
}
}
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
}
}
impl Lines {
/// Create a new `Lines` codec backed by the socket
fn new(socket: TcpStream) -> Self {
Lines {
socket,
rd: BytesMut::new(),
wr: BytesMut::new(),
}
}
/// Buffer a line.
///
/// This writes the line to an internal buffer. Calls to `poll_flush` will
/// attempt to flush this buffer to the socket.
fn buffer(&mut self, line: &[u8]) {
// Ensure the buffer has capacity. Ideally this would not be unbounded,
// but to keep the example simple, we will not limit this.
self.wr.reserve(line.len());
// Push the line onto the end of the write buffer.
//
// The `put` function is from the `BufMut` trait.
self.wr.put(line);
}
/// Flush the write buffer to the socket
fn poll_flush(&mut self) -> Poll<(), io::Error> {
// As long as there is buffered data to write, try to write it.
while !self.wr.is_empty() {
// Try to write some bytes to the socket
let n = try_ready!(self.socket.poll_write(&self.wr));
// As long as the wr is not empty, a successful write should
// never write 0 bytes.
assert!(n > 0);
// This discards the first `n` bytes of the buffer.
let _ = self.wr.split_to(n);
}
Ok(Async::Ready(()))
}
/// Read data from the socket.
///
/// This only returns `Ready` when the socket has closed.
fn fill_read_buf(&mut self) -> Poll<(), io::Error> {
loop {
// Ensure the read buffer has capacity.
//
// This might result in an internal allocation.
self.rd.reserve(1024);
// Read data into the buffer.
let n = try_ready!(self.socket.read_buf(&mut self.rd));
if n == 0 {
return Ok(Async::Ready(()));
}
}
}
}
impl Stream for Lines {
type Item = BytesMut;
type Error = io::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
// First, read any new data that might have been received off the socket
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
if let Some(pos) = pos {
// Remove the line from the read buffer and set it to `line`.
let mut line = self.rd.split_to(pos + 2);
// Drop the trailing \r\n
line.split_off(pos);
// Return the line
return Ok(Async::Ready(Some(line)));
}
if sock_closed {
Ok(Async::Ready(None))
} else {
Ok(Async::NotReady)
}
}
}
/// Spawn a task to manage the socket.
///
/// This will read the first line from the socket to identify the client, then
/// add the client to the set of connected peers in the chat service.
fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// Wrap the socket with the `Lines` codec that we wrote above.
//
// By doing this, we can operate at the line level instead of doing raw byte
// manipulation.
let lines = Lines::new(socket);
// The first line is treated as the client's name. The client is not added
// to the set of connected peers until this line is received.
//
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
// Process the first received line as the client's name.
.and_then(|(name, lines)| {
// If `name` is `None`, then the client disconnected without
// actually sending a line of data.
//
// Since the connection is closed, there is no further work that we
// need to do. So, we just terminate processing by returning
// `future::ok()`.
//
// The problem is that only a single future type can be returned
// from a combinator closure, but we want to return both
// `future::ok()` and `Peer` (below).
//
// This is a common problem, so the `futures` crate solves this by
// providing the `Either` helper enum that allows creating a single
// return type that covers two concrete future types.
let name = match name {
Some(name) => name,
None => {
// The remote client closed the connection without sending
// any data.
return Either::A(future::ok(()));
}
};
println!("`{:?}` is joining the chat", name);
// Create the peer.
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
})
// Task futures have an error of type `()`, this ensures we handle the
// error. We do this by printing the error to STDOUT.
.map_err(|e| {
println!("connection error = {:?}", e);
});
// Spawn the task. Internally, this submits the task to a thread pool.
tokio::spawn(connection);
}
pub fn main() -> Result<(), Box<std::error::Error>> {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse()?;
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr)?;
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
Ok(())
}
-253
View File
@@ -1,253 +0,0 @@
//! An example of hooking up stdin/stdout to either a TCP or UDP stream.
//!
//! This example will connect to a socket address specified in the argument list
//! and then forward all data read on stdin to the server, printing out all data
//! received on stdout. An optional `--udp` argument can be passed to specify
//! that the connection should be made over UDP instead of TCP, translating each
//! line entered on stdin to a UDP packet to be sent to the remote address.
//!
//! Note that this is not currently optimized for performance, especially
//! around buffer management. Rather it's intended to show an example of
//! working with a client.
//!
//! This example can be quite useful when interacting with the other examples in
//! this repository! Many of them recommend running this as a simple "hook up
//! stdin/stdout to a server" to get up and running.
#![deny(warnings)]
extern crate tokio;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use std::env;
use std::io::{self, Read, Write};
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
fn main() -> Result<(), Box<std::error::Error>> {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
Some(i) => {
args.remove(i);
false
}
None => true,
};
// Parse what address we're going to connect to
let addr = match args.first() {
Some(addr) => addr,
None => Err("this program requires at least one argument")?,
};
let addr = addr.parse::<SocketAddr>()?;
// Right now Tokio doesn't support a handle to stdin running on the event
// loop, so we farm out that work to a separate thread. This thread will
// read data (with blocking I/O) from stdin and then send it to the event
// loop over a standard futures channel.
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(|| read_stdin(stdin_tx));
let stdin_rx = stdin_rx.map_err(|_| panic!("errors not possible on rx"));
// Now that we've got our stdin read we either set up our TCP connection or
// our UDP connection to get a stream of bytes we're going to emit to
// stdout.
let stdout = if tcp {
tcp::connect(&addr, Box::new(stdin_rx))?
} else {
udp::connect(&addr, Box::new(stdin_rx))?
};
// And now with our stream of bytes to write to stdout, we execute that in
// the event loop! Note that this is doing blocking I/O to emit data to
// stdout, and in general it's a no-no to do that sort of work on the event
// loop. In this case, though, we know it's ok as the event loop isn't
// otherwise running anything useful.
let mut out = io::stdout();
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
})
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
Ok(())
}
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio::codec::{Encoder, Decoder};
/// A simple `Codec` implementation that just ships bytes around.
///
/// This type is used for "framing" a TCP/UDP stream of bytes but it's really
/// just a convenient method for us to work with streams/sinks for now.
/// This'll just take any data read and interpret it as a "frame" and
/// conversely just shove data into the output location without looking at
/// it.
pub struct Bytes;
impl Decoder for Bytes {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() > 0 {
let len = buf.len();
Ok(Some(buf.split_to(len)))
} else {
Ok(None)
}
}
}
impl Encoder for Bytes {
type Item = Vec<u8>;
type Error = io::Error;
fn encode(&mut self, data: Vec<u8>, buf: &mut BytesMut) -> io::Result<()> {
buf.put(&data[..]);
Ok(())
}
}
}
mod tcp {
use tokio;
use tokio::net::TcpStream;
use tokio::prelude::*;
use tokio::codec::Decoder;
use bytes::BytesMut;
use codec::Bytes;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>>
{
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
// to start forwarding data.
//
// First we use the `Io::framed` method with a simple implementation of
// a `Codec` (listed below) that just ships bytes around. We then split
// that in two to work with the stream and sink separately.
//
// Half of the work we're going to do is to take all data we receive on
// `stdin` and send that along the TCP stream (`sink`). The second half
// is to take all the data we receive (`stream`) and then write that to
// stdout. We'll be passing this handle back out from this method.
//
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
let stream = Box::new(tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream());
Ok(stream)
}
}
mod udp {
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>>
{
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
"0.0.0.0:0".parse()?
} else {
"[::]:0".parse()?
};
let udp = match UdpSocket::bind(&addr_to_bind) {
Ok(udp) => udp,
Err(_) => Err("failed to bind socket")?,
};
// Like above with TCP we use an instance of `Bytes` codec to transform
// this UDP socket into a framed sink/stream which operates over
// discrete values. In this case we're working with *pairs* of socket
// addresses and byte buffers.
let (sink, stream) = UdpFramed::new(udp, Bytes).split();
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
let forward_stdin = stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
let receive = stream.filter_map(move |(chunk, src)| {
if src == addr {
Some(chunk.into())
} else {
None
}
});
let stream = Box::new(future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
}).flatten_stream());
Ok(stream)
}
}
// Our helper method which will read data from stdin and send it along the
// sender provided.
fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
let mut stdin = io::stdin();
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
tx = match tx.send(buf).wait() {
Ok(tx) => tx,
Err(_) => break,
};
}
}
-115
View File
@@ -1,115 +0,0 @@
//! A "hello world" echo server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! write back everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example echo
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be echo'd back to
//! you! If you open up multiple terminals running the `connect` example you
//! should be able to see them all make progress simultaneously.
#![deny(warnings)]
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
fn main() -> Result<(), Box<std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We just want to copy all data read from the socket back onto the
// socket itself (e.g. "echo"). We can use the standard `io::copy`
// combinator in the `tokio-core` crate to do precisely this!
//
// The `copy` function takes two arguments, where to read from and where
// to write to. We only have one argument, though, with `socket`.
// Luckily there's a method, `Io::split`, which will split an Read/Write
// stream into its two halves. This operation allows us to work with
// each stream independently, such as pass them as two arguments to the
// `copy` function.
//
// The `copy` function then returns a future, and this future will be
// resolved when the copying operation is complete, resolving to the
// amount of data that was copied.
let (reader, writer) = socket.split();
let amt = io::copy(reader, writer);
// After our copy operation is complete we just print out some helpful
// information.
let msg = amt.then(move |result| {
match result {
Ok((amt, _, _)) => println!("wrote {} bytes", amt),
Err(e) => println!("error: {}", e),
}
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(msg)
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
-57
View File
@@ -1,57 +0,0 @@
//! Hello world server.
//!
//! A simple client that opens a TCP stream, writes "hello world\n", and closes
//! the connection.
//!
//! You can test this out by running:
//!
//! ncat -l 6142
//!
//! And then in another terminal run:
//!
//! cargo run --example hello_world
#![deny(warnings)]
extern crate tokio;
use tokio::io;
use tokio::net::TcpStream;
use tokio::prelude::*;
pub fn main() -> Result<(), Box<std::error::Error>> {
let addr = "127.0.0.1:6142".parse()?;
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpStream, which is fully async.
let client = TcpStream::connect(&addr).and_then(|stream| {
println!("created stream");
io::write_all(stream, "hello world\n").then(|result| {
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
})
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("connection error = {:?}", err);
});
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
println!("About to create the stream and write to it...");
tokio::run(client);
println!("Stream has been created and written to.");
Ok(())
}
-87
View File
@@ -1,87 +0,0 @@
//! An example how to manually assemble a runtime and run some tasks on it.
//!
//! This is closer to the single-threaded runtime than the default tokio one, as it is simpler to
//! grasp. There are conceptually similar, but the multi-threaded one would be more code. If you
//! just want to *use* a single-threaded runtime, use the one provided by tokio directly
//! (`tokio::runtime::current_thread::Runtime::new()`. This is a demonstration only.
//!
//! Note that the error handling is a bit left out. Also, the `run` could be modified to return the
//! result of the provided future.
extern crate futures;
extern crate tokio;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_timer;
use std::io::Error as IoError;
use std::time::{Duration, Instant};
use futures::{future, Future};
use tokio_current_thread::CurrentThread;
use tokio_reactor::Reactor;
use tokio_timer::timer::{self, Timer};
/// Creates a "runtime".
///
/// This is similar to running `tokio::runtime::current_thread::Runtime::new()`.
fn run<F: Future<Item = (), Error = ()>>(f: F) -> Result<(), IoError> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new(reactor);
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor generate some new stimuli for the
// futures to continue in their life.
let mut executor = CurrentThread::new_with_park(timer);
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the current single-threaded
// executor when used. This is a trick, because we need two mutable references to the
// executor (one to run the provided future, another to install as the default one). We
// use the fake one here as the default one.
let mut default_executor = tokio_current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
// Run the provided future
executor.block_on(f).unwrap();
// Run all the other futures that are still left in the executor
executor.run().unwrap();
});
});
});
Ok(())
}
fn main() -> Result<(), Box<std::error::Error>> {
run(future::lazy(|| {
// Here comes the application logic. It can spawn further tasks by tokio_current_thread::spawn().
// It also can use the default reactor and create timeouts.
// Connect somewhere. And then do nothing with it. Yes, useless.
//
// This will use the default reactor which runs in the current thread.
let connect = tokio::net::TcpStream::connect(&"127.0.0.1:53".parse().unwrap())
.map(|_| println!("Connected"))
.map_err(|e| println!("Failed to connect: {}", e));
// We can spawn it without requiring Send. This would panic if we run it outside of the
// `run` (or outside of anything else)
tokio_current_thread::spawn(connect);
// We can also create timeouts.
let deadline = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(5))
.map(|()| println!("5 seconds are over"))
.map_err(|e| println!("Failed to wait: {}", e));
// We can spawn on the default executor, which is also the local one.
tokio::executor::spawn(deadline);
Ok(())
}))?;
Ok(())
}
-150
View File
@@ -1,150 +0,0 @@
//! A "print-each-packet" server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! put down in the stdout everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example print\_each\_packet
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be written to terminal!
//!
//! Minimal js example:
//!
//! ```js
//! var net = require("net");
//!
//! var listenPort = 8080;
//!
//! var server = net.createServer(function (socket) {
//! socket.on("data", function (bytes) {
//! console.log("bytes", bytes);
//! });
//!
//! socket.on("end", function() {
//! console.log("Socket received FIN packet and closed connection");
//! });
//! socket.on("error", function (error) {
//! console.log("Socket closed with error", error);
//! });
//!
//! socket.on("close", function (with_error) {
//! if (with_error) {
//! console.log("Socket closed with result: Err(SomeError)");
//! } else {
//! console.log("Socket closed with result: Ok(())");
//! }
//! });
//!
//! });
//!
//! server.listen(listenPort);
//!
//! console.log("Listening on:", listenPort);
//! ```
//!
#![deny(warnings)]
extern crate tokio;
extern crate tokio_codec;
use tokio_codec::BytesCodec;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::codec::Decoder;
use std::env;
use std::net::SocketAddr;
fn main() -> Result<(), Box<std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We're parsing each socket with the `BytesCodec` included in `tokio_io`,
// and then we `split` each codec into the reader/writer halves.
//
// See https://docs.rs/tokio-codec/0.1/src/tokio_codec/bytes_codec.rs.html
let framed = BytesCodec::new().framed(socket);
let (_writer, reader) = framed.split();
let processor = reader
.for_each(|bytes| {
println!("bytes: {:?}", bytes);
Ok(())
})
// After our copy operation is complete we just print out some helpful
// information.
.and_then(|()| {
println!("Socket received FIN packet and closed connection");
Ok(())
})
.or_else(|err| {
println!("Socket closed with error: {:?}", err);
// We have to return the error to catch it in the next ``.then` call
Err(err)
})
.then(|result| {
println!("Socket closed with result: {:?}", result);
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(processor)
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
-129
View File
@@ -1,129 +0,0 @@
//! A proxy that forwards data to another server and forwards that server's
//! responses back to clients.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! You can showcase this by running this in one terminal:
//!
//! cargo run --example proxy
//!
//! This in another terminal
//!
//! cargo run --example echo
//!
//! And finally this in another terminal
//!
//! cargo run --example connect 127.0.0.1:8081
//!
//! This final terminal will connect to our proxy, which will in turn connect to
//! the echo server, and you'll be able to see data flowing between them.
#![deny(warnings)]
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use tokio::io::{copy, shutdown};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
fn main() -> Result<(), Box<std::error::Error>> {
let listen_addr = env::args().nth(1).unwrap_or("127.0.0.1:8081".to_string());
let listen_addr = listen_addr.parse::<SocketAddr>()?;
let server_addr = env::args().nth(2).unwrap_or("127.0.0.1:8080".to_string());
let server_addr = server_addr.parse::<SocketAddr>()?;
// Create a TCP listener which will listen for incoming connections.
let socket = TcpListener::bind(&listen_addr)?;
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
let amounts = server.and_then(move |server| {
// Create separate read/write handles for the TCP clients that we're
// proxying data between. Note that typically you'd use
// `AsyncRead::split` for this operation, but we want our writer
// handles to have a custom implementation of `shutdown` which
// actually calls `TcpStream::shutdown` to ensure that EOF is
// transmitted properly across the proxied connection.
//
// As a result, we wrap up our client/server manually in arcs and
// use the impls below on our custom `MyTcpStream` type.
let client_reader = MyTcpStream(Arc::new(Mutex::new(client)));
let client_writer = client_reader.clone();
let server_reader = MyTcpStream(Arc::new(Mutex::new(server)));
let server_writer = server_reader.clone();
// Copy the data (in parallel) between the client and the server.
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
Ok(())
});
tokio::run(done);
Ok(())
}
// This is a custom type used to have a custom implementation of the
// `AsyncWrite::shutdown` method which actually calls `TcpStream::shutdown` to
// notify the remote end that we're done writing.
#[derive(Clone)]
struct MyTcpStream(Arc<Mutex<TcpStream>>);
impl Read for MyTcpStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.0.lock().unwrap().read(buf)
}
}
impl Write for MyTcpStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().unwrap().write(buf)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl AsyncRead for MyTcpStream {}
impl AsyncWrite for MyTcpStream {
fn shutdown(&mut self) -> Poll<(), io::Error> {
try!(self.0.lock().unwrap().shutdown(Shutdown::Write));
Ok(().into())
}
}
-207
View File
@@ -1,207 +0,0 @@
//! A "tiny database" and accompanying protocol
//!
//! This example shows the usage of shared state amongst all connected clients,
//! namely a database of key/value pairs. Each connected client can send a
//! series of GET/SET commands to query the current value of a key or set the
//! value of a key.
//!
//! This example has a simple protocol you can use to interact with the server.
//! To run, first run this in one terminal window:
//!
//! cargo run --example tinydb
//!
//! and next in another windows run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! In the `connect` window you can type in commands where when you hit enter
//! you'll get a response from the server for that command. An example session
//! is:
//!
//!
//! $ cargo run --example connect 127.0.0.1:8080
//! GET foo
//! foo = bar
//! GET FOOBAR
//! error: no key FOOBAR
//! SET FOOBAR my awesome string
//! set FOOBAR = `my awesome string`, previous: None
//! SET foo tokio
//! set foo = `tokio`, previous: Some("bar")
//! GET foo
//! foo = tokio
//!
//! Namely you can issue two forms of commands:
//!
//! * `GET $key` - this will fetch the value of `$key` from the database and
//! return it. The server's database is initially populated with the key `foo`
//! set to the value `bar`
//! * `SET $key $value` - this will set the value of `$key` to `$value`,
//! returning the previous value, if any.
#![deny(warnings)]
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use tokio::io::{lines, write_all};
use tokio::net::TcpListener;
use tokio::prelude::*;
/// The in-memory database shared amongst all clients.
///
/// This database will be shared via `Arc`, so to mutate the internal map we're
/// going to use a `Mutex` for interior mutability.
struct Database {
map: Mutex<HashMap<String, String>>,
}
/// Possible requests our clients can send us
enum Request {
Get { key: String },
Set { key: String, value: String },
}
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
}
fn main() -> Result<(), Box<std::error::Error>> {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr).map_err(|_| "failed to bind")?;
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
// clients. We populate the initial database and then create the `Database`
// structure. Note the usage of `Arc` here which will be used to ensure that
// each independently spawned client will have a reference to the in-memory
// database.
let mut initial_db = HashMap::new();
initial_db.insert("foo".to_string(), "bar".to_string());
let db = Arc::new(Database {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
// *split* this TCP stream into two separately owned halves. This'll
// allow us to work with the read and write halves independently.
let (reader, writer) = socket.split();
// Since our protocol is line-based we use `tokio_io`'s `lines` utility
// to convert our stream of bytes, `reader`, into a `Stream` of lines.
let lines = lines(BufReader::new(reader));
// Here's where the meat of the processing in this server happens. First
// we see a clone of the database being created, which is creating a
// new reference for this connected client to use. Also note the `move`
// keyword on the closure here which moves ownership of the reference
// into the closure, which we'll need for spawning the client below.
//
// The `map` function here means that we'll run some code for all
// requests (lines) we receive from the client. The actual handling here
// is pretty simple, first we parse the request and if it's valid we
// generate a response based on the values in the database.
let db = db.clone();
let responses = lines.map(move |line| {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
}
}
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
}
}
});
// At this point `responses` is a stream of `Response` types which we
// now want to write back out to the client. To do that we use
// `Stream::fold` to perform a loop here, serializing each response and
// then writing it out to the client.
let writes = responses.fold(writer, |writer, response| {
let mut response = response.serialize();
response.push('\n');
write_all(writer, response.into_bytes()).map(|(w, _)| w)
});
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients, for now ignoring any errors
// that we see.
let msg = writes.then(move |_| Ok(()));
tokio::spawn(msg)
});
tokio::run(done);
Ok(())
}
impl Request {
fn parse(input: &str) -> Result<Request, String> {
let mut parts = input.splitn(3, " ");
match parts.next() {
Some("GET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err(format!("GET must be followed by a key")),
};
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
}
Ok(Request::Get { key: key.to_string() })
}
Some("SET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err(format!("SET must be followed by a key")),
};
let value = match parts.next() {
Some(value) => value,
None => return Err(format!("SET needs a value")),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
}
}
}
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
}
}
}
-65
View File
@@ -1,65 +0,0 @@
//! This example leverages `BytesCodec` to create a UDP client and server which
//! speak a custom protocol.
//!
//! Here we're using the codec from tokio-io to convert a UDP socket to a stream of
//! client messages. These messages are then processed and returned back as a
//! new message with a new destination. Overall, we then use this to construct a
//! "ping pong" pair where two sockets are sending messages back and forth.
#![deny(warnings)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate env_logger;
use std::net::SocketAddr;
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_codec::BytesCodec;
fn main() -> Result<(), Box<std::error::Error>> {
let _ = env_logger::init();
let addr: SocketAddr = "127.0.0.1:0".parse()?;
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr)?;
let b = UdpSocket::bind(&addr)?;
let b_addr = b.local_addr()?;
// We're parsing each socket with the `BytesCodec` included in `tokio_io`, and then we
// `split` each codec into the sink/stream halves.
let (a_sink, a_stream) = UdpFramed::new(a, BytesCodec::new()).split();
let (b_sink, b_stream) = UdpFramed::new(b, BytesCodec::new()).split();
// Start off by sending a ping from a to b, afterwards we just print out
// what they send us and continually send pings
// let pings = stream::iter((0..5).map(Ok));
let a = a_sink.send(("PING".into(), b_addr)).and_then(|a_sink| {
let mut i = 0;
let a_stream = a_stream.take(4).map(move |(msg, addr)| {
i += 1;
println!("[a] recv: {}", String::from_utf8_lossy(&msg));
(format!("PING {}", i).into(), addr)
});
a_sink.send_all(a_stream)
});
// The second client we have will receive the pings from `a` and then send
// back pongs.
let b_stream = b_stream.map(|(msg, addr)| {
println!("[b] recv: {}", String::from_utf8_lossy(&msg));
("PONG".into(), addr)
});
let b = b_sink.send_all(b_stream);
// Spawn the sender of pongs and then wait for our pinger to finish.
tokio::run({
b.join(a)
.map(|_| ())
.map_err(|e| println!("error = {:?}", e))
});
Ok(())
}
+1
View File
@@ -0,0 +1 @@
nightly-2019-08-21
+1
View File
@@ -0,0 +1 @@
edition = "2018"
-26
View File
@@ -1,26 +0,0 @@
use std::future::{Future as StdFuture};
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
-170
View File
@@ -1,170 +0,0 @@
#![allow(deprecated)]
//! Execute many tasks concurrently on the current thread.
//!
//! [`CurrentThread`] is an executor that keeps tasks on the same thread that
//! they were spawned from. This allows it to execute futures that are not
//! `Send`.
//!
//! A single [`CurrentThread`] instance is able to efficiently manage a large
//! number of tasks and will attempt to schedule all tasks fairly.
//!
//! All tasks that are being managed by a [`CurrentThread`] executor are able to
//! spawn additional tasks by calling [`spawn`]. This function only works from
//! within the context of a running [`CurrentThread`] instance.
//!
//! The easiest way to start a new [`CurrentThread`] executor is to call
//! [`block_on_all`] with an initial task to seed the executor.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread;
//! use futures::future::lazy;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! current_thread::block_on_all(lazy(|| {
//! // The execution context is setup, futures may be executed.
//! current_thread::spawn(lazy(|| {
//! println!("called from the current thread executor");
//! Ok(())
//! }));
//!
//! Ok::<_, ()>(())
//! }));
//! # }
//! ```
//!
//! The `block_on_all` function will block the current thread until **all**
//! tasks that have been spawned onto the [`CurrentThread`] instance have
//! completed.
//!
//! More fine-grain control can be achieved by using [`CurrentThread`] directly.
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread::CurrentThread;
//! use futures::future::{lazy, empty};
//! use std::time::Duration;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! let mut current_thread = CurrentThread::new();
//!
//! // Spawn a task, the task is not executed yet.
//! current_thread.spawn(lazy(|| {
//! println!("Spawning a task");
//! Ok(())
//! }));
//!
//! // Spawn a task that never completes
//! current_thread.spawn(empty());
//!
//! // Run the executor, but only until the provided future completes. This
//! // provides the opportunity to start executing previously spawned tasks.
//! let res = current_thread.block_on(lazy(|| {
//! Ok::<_, ()>("Hello")
//! })).unwrap();
//!
//! // Now, run the executor for *at most* 1 second. Since a task was spawned
//! // that never completes, this function will return with an error.
//! current_thread.run_timeout(Duration::from_secs(1)).unwrap_err();
//! # }
//! ```
//!
//! # Execution model
//!
//! Internally, [`CurrentThread`] maintains a queue. When one of its tasks is
//! notified, the task gets added to the queue. The executor will pop tasks from
//! the queue and call [`Future::poll`]. If the task gets notified while it is
//! being executed, it won't get re-executed until all other tasks currently in
//! the queue get polled.
//!
//! Before the task is polled, a thread-local variable referencing the current
//! [`CurrentThread`] instance is set. This enables [`spawn`] to spawn new tasks
//! onto the same executor without having to thread through a handle value.
//!
//! If the [`CurrentThread`] instance still has uncompleted tasks, but none of
//! these tasks are ready to be polled, the current thread is put to sleep. When
//! a task is notified, the thread is woken up and processing resumes.
//!
//! All tasks managed by [`CurrentThread`] remain on the current thread. When a
//! task completes, it is dropped.
//!
//! [`spawn`]: fn.spawn.html
//! [`block_on_all`]: fn.block_on_all.html
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
pub use tokio_current_thread::{
BlockError,
CurrentThread,
Entered,
Handle,
RunError,
RunTimeoutError,
TaskExecutor,
Turn,
TurnError,
block_on_all,
spawn,
};
use std::cell::Cell;
use std::marker::PhantomData;
use futures::future::{self};
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[derive(Debug)]
pub struct Context<'a> {
cancel: Cell<bool>,
_p: PhantomData<&'a ()>,
}
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
pub fn run<F, R>(f: F) -> R
where F: FnOnce(&mut Context) -> R
{
let mut context = Context {
cancel: Cell::new(false),
_p: PhantomData,
};
let mut current_thread = CurrentThread::new();
let ret = current_thread
.block_on(future::lazy(|| Ok::<_, ()>(f(&mut context))))
.unwrap();
if context.cancel.get() {
return ret;
}
current_thread.run().unwrap();
ret
}
#[deprecated(since = "0.1.2", note = "use TaskExecutor::current instead")]
#[doc(hidden)]
pub fn task_executor() -> TaskExecutor {
TaskExecutor::current()
}
-155
View File
@@ -1,155 +0,0 @@
#![doc(html_root_url = "https://docs.rs/tokio/0.1.15")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
#![cfg_attr(feature = "async-await-preview", feature(
async_await,
await_macro,
futures_api,
))]
//! A runtime for writing reliable, asynchronous, and slim applications.
//!
//! Tokio is an event-driven, non-blocking I/O platform for writing asynchronous
//! applications with the Rust programming language. At a high level, it
//! provides a few major components:
//!
//! * A multi threaded, work-stealing based task [scheduler][runtime].
//! * A [reactor] backed by the operating system's event queue (epoll, kqueue,
//! IOCP, etc...).
//! * Asynchronous [TCP and UDP][net] sockets.
//! * Asynchronous [filesystem][fs] operations.
//! * [Timer][timer] API for scheduling work in the future.
//!
//! Tokio is built using [futures] as the abstraction for managing the
//! complexity of asynchronous programming.
//!
//! Guide level documentation is found on the [website].
//!
//! [website]: https://tokio.rs/docs/getting-started/hello-world/
//! [futures]: http://docs.rs/futures/0.1
//!
//! # Examples
//!
//! A simple TCP echo server:
//!
//! ```no_run
//! extern crate tokio;
//!
//! use tokio::prelude::*;
//! use tokio::io::copy;
//! use tokio::net::TcpListener;
//!
//! fn main() {
//! // Bind the server's socket.
//! let addr = "127.0.0.1:12345".parse().unwrap();
//! let listener = TcpListener::bind(&addr)
//! .expect("unable to bind TCP listener");
//!
//! // Pull out a stream of sockets for incoming connections
//! let server = listener.incoming()
//! .map_err(|e| eprintln!("accept failed = {:?}", e))
//! .for_each(|sock| {
//! // Split up the reading and writing parts of the
//! // socket.
//! let (reader, writer) = sock.split();
//!
//! // A future that echos the data and returns how
//! // many bytes were copied...
//! let bytes_copied = copy(reader, writer);
//!
//! // ... after which we'll print what happened.
//! let handle_conn = bytes_copied.map(|amt| {
//! println!("wrote {:?} bytes", amt)
//! }).map_err(|err| {
//! eprintln!("IO error {:?}", err)
//! });
//!
//! // Spawn the future as a concurrent task.
//! tokio::spawn(handle_conn)
//! });
//!
//! // Start the Tokio runtime
//! tokio::run(server);
//! }
//! ```
macro_rules! if_runtime {
($($i:item)*) => ($(
#[cfg(any(feature = "rt-full"))]
$i
)*)
}
#[macro_use]
extern crate futures;
#[cfg(feature = "io")]
extern crate bytes;
#[cfg(feature = "reactor")]
extern crate mio;
#[cfg(feature = "rt-full")]
extern crate num_cpus;
#[cfg(feature = "rt-full")]
extern crate tokio_current_thread;
#[cfg(feature = "io")]
extern crate tokio_io;
#[cfg(feature = "codec")]
extern crate tokio_codec;
#[cfg(feature = "fs")]
extern crate tokio_fs;
#[cfg(feature = "reactor")]
extern crate tokio_reactor;
#[cfg(feature = "rt-full")]
extern crate tokio_threadpool;
#[cfg(feature = "sync")]
extern crate tokio_sync;
#[cfg(feature = "timer")]
extern crate tokio_timer;
#[cfg(feature = "tcp")]
extern crate tokio_tcp;
#[cfg(feature = "udp")]
extern crate tokio_udp;
#[cfg(feature = "async-await-preview")]
extern crate tokio_async_await;
#[cfg(all(unix, feature = "uds"))]
extern crate tokio_uds;
#[cfg(feature = "timer")]
pub mod clock;
#[cfg(feature = "codec")]
pub mod codec;
#[cfg(feature = "fs")]
pub mod fs;
#[cfg(feature = "io")]
pub mod io;
#[cfg(any(feature = "tcp", feature = "udp", feature = "uds"))]
pub mod net;
pub mod prelude;
#[cfg(feature = "reactor")]
pub mod reactor;
#[cfg(feature = "sync")]
pub mod sync;
#[cfg(feature = "timer")]
pub mod timer;
pub mod util;
if_runtime! {
extern crate tokio_executor;
pub mod executor;
pub mod runtime;
pub use executor::spawn;
pub use runtime::run;
}
// ===== Experimental async/await support =====
#[cfg(feature = "async-await-preview")]
mod async_await;
#[cfg(feature = "async-await-preview")]
pub use async_await::{run_async, spawn_async};
#[cfg(feature = "async-await-preview")]
pub use tokio_async_await::await;
-55
View File
@@ -1,55 +0,0 @@
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
#[cfg(feature = "io")]
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
pub use util::{
FutureExt,
StreamExt,
};
pub use ::std::io::{
Read,
Write,
};
pub use futures::{
Future,
future,
Stream,
stream,
Sink,
IntoFuture,
Async,
AsyncSink,
Poll,
task,
};
#[cfg(feature = "async-await-preview")]
#[doc(inline)]
pub use tokio_async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt as StreamAsyncExt,
},
};
-539
View File
@@ -1,539 +0,0 @@
//! Readiness tracking streams, backing I/O objects.
//!
//! This module contains the core type which is used to back all I/O on object
//! in `tokio-core`. The `PollEvented` type is the implementation detail of
//! all I/O. Each `PollEvented` manages registration with a reactor,
//! acquisition of a token, and tracking of the readiness state on the
//! underlying I/O primitive.
#![allow(deprecated, warnings)]
use std::fmt;
use std::io::{self, Read, Write};
use std::sync::Mutex;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::{task, Async, Poll};
use mio::event::Evented;
use mio::Ready;
use tokio_io::{AsyncRead, AsyncWrite};
use reactor::{Handle, Registration};
#[deprecated(since = "0.1.2", note = "PollEvented2 instead")]
#[doc(hidden)]
pub struct PollEvented<E> {
io: E,
inner: Inner,
handle: Handle,
}
struct Inner {
registration: Mutex<Registration>,
/// Currently visible read readiness
read_readiness: AtomicUsize,
/// Currently visible write readiness
write_readiness: AtomicUsize,
}
impl<E: fmt::Debug> fmt::Debug for PollEvented<E> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("PollEvented")
.field("io", &self.io)
.finish()
}
}
impl<E> PollEvented<E> {
/// Creates a new readiness stream associated with the provided
/// `loop_handle` and for the given `source`.
pub fn new(io: E, handle: &Handle) -> io::Result<PollEvented<E>>
where E: Evented,
{
let registration = Registration::new();
registration.register(&io)?;
Ok(PollEvented {
io: io,
inner: Inner {
registration: Mutex::new(registration),
read_readiness: AtomicUsize::new(0),
write_readiness: AtomicUsize::new(0),
},
handle: handle.clone(),
})
}
/// Tests to see if this source is ready to be read from or not.
///
/// If this stream is not ready for a read then `Async::NotReady` will be
/// returned and the current task will be scheduled to receive a
/// notification when the stream is readable again. In other words, this
/// method is only safe to call from within the context of a future's task,
/// typically done in a `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::readable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_read(&mut self) -> Async<()> {
if self.poll_read2().is_ready() {
return ().into();
}
Async::NotReady
}
fn poll_read2(&self) -> Async<Ready> {
let r = self.inner.registration.lock().unwrap();
// Load the cached readiness
match self.inner.read_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_read_ready().unwrap() {
n |= ready2usize(ready);
self.inner.read_readiness.store(n, Relaxed);
}
return usize2ready(n).into();
}
}
let ready = match r.poll_read_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.read_readiness.store(ready2usize(ready), Relaxed);
ready.into()
}
/// Tests to see if this source is ready to be written to or not.
///
/// If this stream is not ready for a write then `Async::NotReady` will be returned
/// and the current task will be scheduled to receive a notification when
/// the stream is writable again. In other words, this method is only safe
/// to call from within the context of a future's task, typically done in a
/// `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::writable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_write(&mut self) -> Async<()> {
let r = self.inner.registration.lock().unwrap();
match self.inner.write_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_write_ready().unwrap() {
n |= ready2usize(ready);
self.inner.write_readiness.store(n, Relaxed);
}
return ().into();
}
}
let ready = match r.poll_write_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.write_readiness.store(ready2usize(ready), Relaxed);
().into()
}
/// Test to see whether this source fulfills any condition listed in `mask`
/// provided.
///
/// The `mask` given here is a mio `Ready` set of possible events. This can
/// contain any events like read/write but also platform-specific events
/// such as hup and error. The `mask` indicates events that are interested
/// in being ready.
///
/// If any event in `mask` is ready then it is returned through
/// `Async::Ready`. The `Ready` set returned is guaranteed to not be empty
/// and contains all events that are currently ready in the `mask` provided.
///
/// If no events are ready in the `mask` provided then the current task is
/// scheduled to receive a notification when any of them become ready. If
/// the `writable` event is contained within `mask` then this
/// `PollEvented`'s `write` task will be blocked and otherwise the `read`
/// task will be blocked. This is generally only relevant if you're working
/// with this `PollEvented` object on multiple tasks.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_ready(&mut self, mask: Ready) -> Async<Ready> {
let mut ret = Ready::empty();
if mask.is_empty() {
return ret.into();
}
if mask.is_writable() {
if self.poll_write().is_ready() {
ret = Ready::writable();
}
}
let mask = mask - Ready::writable();
if !mask.is_empty() {
if let Async::Ready(v) = self.poll_read2() {
ret |= v & mask;
}
}
if ret.is_empty() {
if mask.is_writable() {
let _ = self.need_write();
}
if mask.is_readable() {
let _ = self.need_read();
}
Async::NotReady
} else {
ret.into()
}
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer readable, but it needs to be.
///
/// This function, like `poll_read`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// readable, typically because a "would block" error was seen.
///
/// *All* readiness bits associated with this stream except the writable bit
/// will be reset when this method is called. The current task is then
/// scheduled to receive a notification whenever anything changes other than
/// the writable bit. Note that this typically just means the readable bit
/// is used here, but if you're using a custom I/O object for events like
/// hup/error this may also be relevant.
///
/// Note that it is also only valid to call this method if `poll_read`
/// previously indicated that the object is readable. That is, this function
/// must always be paired with calls to `poll_read` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_read(&mut self) -> io::Result<()> {
self.inner.read_readiness.store(0, Relaxed);
if self.poll_read().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer writable, but it needs to be.
///
/// This function, like `poll_write`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// writable, typically because a "would block" error was seen.
///
/// The flag indicating that this stream is writable is unset and the
/// current task is scheduled to receive a notification when the stream is
/// then again writable.
///
/// Note that it is also only valid to call this method if `poll_write`
/// previously indicated that the object is writable. That is, this function
/// must always be paired with calls to `poll_write` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_write(&mut self) -> io::Result<()> {
self.inner.write_readiness.store(0, Relaxed);
if self.poll_write().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Returns a reference to the event loop handle that this readiness stream
/// is associated with.
pub fn handle(&self) -> &Handle {
&self.handle
}
/// Returns a shared reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_ref(&self) -> &E {
&self.io
}
/// Returns a mutable reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_mut(&mut self) -> &mut E {
&mut self.io
}
/// Consumes the `PollEvented` and returns the underlying I/O object
pub fn into_inner(self) -> E {
self.io
}
/// Deregisters this source of events from the reactor core specified.
///
/// This method can optionally be called to unregister the underlying I/O
/// object with the event loop that the `handle` provided points to.
/// Typically this method is not required as this automatically happens when
/// `E` is dropped, but for some use cases the `E` object doesn't represent
/// an owned reference, so dropping it won't automatically unregister with
/// the event loop.
///
/// This consumes `self` as it will no longer provide events after the
/// method is called, and will likely return an error if this `PollEvented`
/// was created on a separate event loop from the `handle` specified.
pub fn deregister(&self) -> io::Result<()>
where E: Evented,
{
self.inner.registration.lock().unwrap()
.deregister(&self.io)
}
}
impl<E: Read> Read for PollEvented<E> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_read() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().read(buf);
if is_wouldblock(&r) {
self.need_read()?;
}
return r
}
}
impl<E: Write> Write for PollEvented<E> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().write(buf);
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
fn flush(&mut self) -> io::Result<()> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().flush();
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
}
impl<E: Read> AsyncRead for PollEvented<E> {
}
impl<E: Write> AsyncWrite for PollEvented<E> {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
fn is_wouldblock<T>(r: &io::Result<T>) -> bool {
match *r {
Ok(_) => false,
Err(ref e) => e.kind() == io::ErrorKind::WouldBlock,
}
}
const READ: usize = 1 << 0;
const WRITE: usize = 1 << 1;
fn ready2usize(ready: Ready) -> usize {
let mut bits = 0;
if ready.is_readable() {
bits |= READ;
}
if ready.is_writable() {
bits |= WRITE;
}
bits | platform::ready2usize(ready)
}
fn usize2ready(bits: usize) -> Ready {
let mut ready = Ready::empty();
if bits & READ != 0 {
ready.insert(Ready::readable());
}
if bits & WRITE != 0 {
ready.insert(Ready::writable());
}
ready | platform::usize2ready(bits)
}
#[cfg(unix)]
mod platform {
use mio::Ready;
use mio::unix::UnixReady;
const HUP: usize = 1 << 2;
const ERROR: usize = 1 << 3;
const AIO: usize = 1 << 4;
const LIO: usize = 1 << 5;
#[cfg(any(target_os = "dragonfly", target_os = "freebsd"))]
fn is_aio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_aio()
}
#[cfg(not(any(target_os = "dragonfly", target_os = "freebsd")))]
fn is_aio(_ready: &Ready) -> bool {
false
}
#[cfg(target_os = "freebsd")]
fn is_lio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_lio()
}
#[cfg(not(target_os = "freebsd"))]
fn is_lio(_ready: &Ready) -> bool {
false
}
pub fn ready2usize(ready: Ready) -> usize {
let ready = UnixReady::from(ready);
let mut bits = 0;
if is_aio(&ready) {
bits |= AIO;
}
if is_lio(&ready) {
bits |= LIO;
}
if ready.is_error() {
bits |= ERROR;
}
if ready.is_hup() {
bits |= HUP;
}
bits
}
#[cfg(any(target_os = "dragonfly", target_os = "freebsd", target_os = "ios",
target_os = "macos"))]
fn usize2ready_aio(ready: &mut UnixReady) {
ready.insert(UnixReady::aio());
}
#[cfg(not(any(target_os = "dragonfly",
target_os = "freebsd", target_os = "ios", target_os = "macos")))]
fn usize2ready_aio(_ready: &mut UnixReady) {
// aio not available here → empty
}
#[cfg(target_os = "freebsd")]
fn usize2ready_lio(ready: &mut UnixReady) {
ready.insert(UnixReady::lio());
}
#[cfg(not(target_os = "freebsd"))]
fn usize2ready_lio(_ready: &mut UnixReady) {
// lio not available here → empty
}
pub fn usize2ready(bits: usize) -> Ready {
let mut ready = UnixReady::from(Ready::empty());
if bits & AIO != 0 {
usize2ready_aio(&mut ready);
}
if bits & LIO != 0 {
usize2ready_lio(&mut ready);
}
if bits & HUP != 0 {
ready.insert(UnixReady::hup());
}
if bits & ERROR != 0 {
ready.insert(UnixReady::error());
}
ready.into()
}
}
#[cfg(windows)]
mod platform {
use mio::Ready;
pub fn all() -> Ready {
// No platform-specific Readinesses for Windows
Ready::empty()
}
pub fn hup() -> Ready {
Ready::empty()
}
pub fn ready2usize(_r: Ready) -> usize {
0
}
pub fn usize2ready(_r: usize) -> Ready {
Ready::empty()
}
}
-125
View File
@@ -1,125 +0,0 @@
//! A batteries included runtime for applications using Tokio.
//!
//! Applications using Tokio require some runtime support in order to work:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
//!
//! [`Runtime`] bundles all of these various runtime components into a single
//! handle that can be started and shutdown together, eliminating the necessary
//! boilerplate to run a Tokio application.
//!
//! Most applications wont need to use [`Runtime`] directly. Instead, they will
//! use the [`run`] function, which uses [`Runtime`] under the hood.
//!
//! Creating a [`Runtime`] does the following:
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//! * Run an instance of [`Timer`] **per** thread pool worker thread.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! A timer per thread pool worker thread is used to minimize the amount of
//! synchronization that is required for working with the timer.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
//! "seed" the application, blocking the thread until the runtime becomes
//! [idle].
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! tokio::run(server);
//! # }
//! # pub fn main() {}
//! ```
//!
//! In this function, the `run` function blocks until the runtime becomes idle.
//! See [`shutdown_on_idle`][idle] for more shutdown details.
//!
//! From within the context of the runtime, additional tasks are spawned using
//! the [`tokio::spawn`] function. Futures spawned using this function will be
//! executed on the same thread pool used by the [`Runtime`].
//!
//! A [`Runtime`] instance can also be used directly.
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::runtime::Runtime;
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! // Create the runtime
//! let mut rt = Runtime::new().unwrap();
//!
//! // Spawn the server task
//! rt.spawn(server);
//!
//! // Wait until the runtime becomes idle and shut it down.
//! rt.shutdown_on_idle()
//! .wait().unwrap();
//! # }
//! # pub fn main() {}
//! ```
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../timer/index.html
//! [`Runtime`]: struct.Runtime.html
//! [`Reactor`]: ../reactor/struct.Reactor.html
//! [`ThreadPool`]: https://docs.rs/tokio-threadpool/0.1/tokio_threadpool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
//! [`Timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/timer/struct.Timer.html
pub mod current_thread;
mod threadpool;
pub use self::threadpool::{
Builder,
Runtime,
Shutdown,
TaskExecutor,
run,
};
-395
View File
@@ -1,395 +0,0 @@
mod builder;
mod shutdown;
mod task_executor;
pub use self::builder::Builder;
pub use self::shutdown::Shutdown;
pub use self::task_executor::TaskExecutor;
use reactor::{Handle, Reactor};
use std::io;
use std::sync::Mutex;
use tokio_executor::enter;
use tokio_threadpool as threadpool;
use futures;
use futures::future::Future;
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// Instances of `Runtime` can be created using [`new`] or [`Builder`]. However,
/// most users will use [`tokio::run`], which uses a `Runtime` internally.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
/// [`new`]: #method.new
/// [`Builder`]: struct.Builder.html
/// [`tokio::run`]: fn.run.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
#[derive(Debug)]
struct Inner {
/// A handle to the reactor in the background thread.
reactor_handle: Handle,
// TODO: This should go away in 0.2
reactor: Mutex<Option<Reactor>>,
/// Task execution pool.
pool: threadpool::ThreadPool,
}
// ===== impl Runtime =====
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// This function is used to bootstrap the execution of a Tokio application. It
/// does the following:
///
/// * Start the Tokio runtime using a default configuration.
/// * Spawn the given future onto the thread pool.
/// * Block the current thread until the runtime shuts down.
///
/// Note that the function will not return immediately once `future` has
/// completed. Instead it waits for the entire runtime to become idle.
///
/// See the [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if called from the context of an executor.
///
/// [mod]: ../index.html
pub fn run<F>(future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
// Check enter before creating a new Runtime...
let mut entered = enter().expect("nested tokio::run");
let mut runtime = Runtime::new().expect("failed to start new Runtime");
runtime.spawn(future);
entered
.block_on(runtime.shutdown_on_idle())
.expect("shutdown cannot error")
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// This results in a reactor, thread pool, and timer being initialized. The
/// thread pool will not spawn any worker threads until it needs to, i.e.
/// tasks are scheduled to run.
///
/// Most users will not need to call this function directly, instead they
/// will use [`tokio::run`](fn.run.html).
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// Creating a new `Runtime` with default configuration values.
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
Builder::new().build()
}
#[deprecated(since = "0.1.5", note = "use `reactor` instead")]
#[doc(hidden)]
pub fn handle(&self) -> &Handle {
#[allow(deprecated)]
self.reactor()
}
/// Return a reference to the reactor handle for this runtime instance.
///
/// The returned handle reference can be cloned in order to get an owned
/// value of the handle. This handle can be used to initialize I/O resources
/// (like TCP or UDP sockets) that will not be used on the runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let reactor_handle = rt.reactor().clone();
///
/// // use `reactor_handle`
/// ```
#[deprecated(since = "0.1.11", note = "there is now a reactor per worker thread")]
pub fn reactor(&self) -> &Handle {
let mut reactor = self.inner().reactor.lock().unwrap();
if let Some(reactor) = reactor.take() {
if let Ok(background) = reactor.background() {
background.forget();
}
}
&self.inner().reactor_handle
}
/// Return a handle to the runtime's executor.
///
/// The returned handle can be used to spawn tasks that run on this runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let executor_handle = rt.executor();
///
/// // use `executor_handle`
/// ```
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
}
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner_mut().pool.sender().spawn(future).unwrap();
self
}
/// Run a future to completion on the Tokio runtime.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, and yielding its resolved result. Any tasks or timers which
/// the future spawns internally will be executed on the runtime.
///
/// This method should not be called from an asynchronous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on<F, R, E>(&mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let mut entered = enter().expect("nested block_on");
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
entered.block_on(rx).unwrap()
}
/// Run a future to completion on the Tokio runtime, then wait for all
/// background futures to complete too.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, waiting for background futures to complete, and yielding
/// its resolved result. Any tasks or timers which the future spawns
/// internally will be executed on the runtime and waited for completion.
///
/// This method should not be called from an asynchronous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on_all<F, R, E>(mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let mut entered = enter().expect("nested block_on_all");
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
let block = rx
.map_err(|_| unreachable!())
.and_then(move |r| {
self.shutdown_on_idle()
.map(move |()| r)
});
entered.block_on(block).unwrap()
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function can be used to perform a graceful shutdown of the runtime.
///
/// The runtime enters an idle state once **all** of the following occur.
///
/// * The thread pool has no tasks to execute, i.e., all tasks that were
/// spawned have completed.
/// * The reactor is not managing any I/O resources.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_on_idle()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = inner.pool.shutdown_on_idle();
Shutdown { inner }
}
/// Signals the runtime to shutdown immediately.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function will forcibly shutdown the runtime, causing any
/// in-progress work to become canceled. The shutdown steps are:
///
/// * Drain any scheduled work queues.
/// * Drop any futures that have not yet completed.
/// * Drop the reactor.
///
/// Once the reactor has dropped, any outstanding I/O resources bound to
/// that reactor will no longer function. Calling any method on them will
/// result in an error.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
Shutdown::shutdown_now(inner)
}
fn inner(&self) -> &Inner {
self.inner.as_ref().unwrap()
}
fn inner_mut(&mut self) -> &mut Inner {
self.inner.as_mut().unwrap()
}
}
impl Drop for Runtime {
fn drop(&mut self) {
if let Some(inner) = self.inner.take() {
let shutdown = Shutdown::shutdown_now(inner);
let _ = shutdown.wait();
}
}
}
-36
View File
@@ -1,36 +0,0 @@
use super::Inner;
use tokio_threadpool as threadpool;
use std::fmt;
use futures::{Future, Poll};
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
pub(super) inner: threadpool::Shutdown,
}
impl Shutdown {
pub(super) fn shutdown_now(inner: Inner) -> Self {
let inner = inner.pool.shutdown_now();
Shutdown { inner }
}
}
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
-80
View File
@@ -1,80 +0,0 @@
use futures::{Async, Poll, Stream, Sink, StartSend};
/// A stream combinator which combines the yields the current item
/// plus its count starting from 0.
///
/// This structure is produced by the `Stream::enumerate` method.
#[derive(Debug)]
#[must_use = "Does nothing unless polled"]
pub struct Enumerate<T> {
inner: T,
count: usize,
}
impl<T> Enumerate<T> {
pub(crate) fn new(stream: T) -> Self {
Self { inner: stream, count: 0 }
}
/// Acquires a reference to the underlying stream that this combinator is
/// pulling from.
pub fn get_ref(&self) -> &T {
&self.inner
}
/// Acquires a mutable reference to the underlying stream that this
/// combinator is pulling from.
///
/// Note that care must be taken to avoid tampering with the state of the
/// stream which may otherwise confuse this combinator.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
/// Consumes this combinator, returning the underlying stream.
///
/// Note that this may discard intermediate state of this combinator, so
/// care should be taken to avoid losing resources when this is called.
pub fn into_inner(self) -> T {
self.inner
}
}
impl<T> Stream for Enumerate<T>
where
T: Stream,
{
type Item = (usize, T::Item);
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, T::Error> {
match try_ready!(self.inner.poll()) {
Some(item) => {
let ret = Some((self.count, item));
self.count += 1;
Ok(Async::Ready(ret))
}
None => return Ok(Async::Ready(None)),
}
}
}
// Forwarding impl of Sink from the underlying stream
impl<T> Sink for Enumerate<T>
where T: Sink
{
type SinkItem = T::SinkItem;
type SinkError = T::SinkError;
fn start_send(&mut self, item: T::SinkItem) -> StartSend<T::SinkItem, T::SinkError> {
self.inner.start_send(item)
}
fn poll_complete(&mut self) -> Poll<(), T::SinkError> {
self.inner.poll_complete()
}
fn close(&mut self) -> Poll<(), T::SinkError> {
self.inner.close()
}
}
-15
View File
@@ -1,15 +0,0 @@
//! Utilities for working with Tokio.
//!
//! This module contains utilities that are useful for working with Tokio.
//! Currently, this only includes [`FutureExt`] and [`StreamExt`], but this
//! may grow over time.
//!
//! [`FutureExt`]: trait.FutureExt.html
//! [`StreamExt`]: trait.StreamExt.html
mod future;
mod stream;
mod enumerate;
pub use self::future::FutureExt;
pub use self::stream::StreamExt;
-63
View File
@@ -1,63 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::net::TcpStream;
use std::thread;
use std::io::{Read, Write, BufReader, BufWriter};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::copy;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
const N: usize = 1024;
drop(env_logger::try_init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let t2 = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let mut b = vec![0; msg.len() * N];
t!(s.read_exact(&mut b));
b
});
let mut expected = Vec::<u8>::new();
for _i in 0..N {
expected.extend(msg.as_bytes());
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
}
(expected, t2)
});
let clients = srv.incoming().take(2).collect();
let copied = clients.and_then(|clients| {
let mut clients = clients.into_iter();
let a = BufReader::new(clients.next().unwrap());
let b = BufWriter::new(clients.next().unwrap());
copy(a, b)
});
let (amt, _, _) = t!(copied.wait());
let (expected, t2) = t.join().unwrap();
let actual = t2.join().unwrap();
assert!(expected == actual);
assert_eq!(amt, msg.len() as u64 * 1024);
}
-69
View File
@@ -1,69 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_timer;
extern crate env_logger;
use tokio::prelude::*;
use tokio::runtime::{self, current_thread};
use tokio::timer::*;
use tokio_timer::clock::Clock;
use std::sync::mpsc;
use std::time::{Duration, Instant};
struct MockNow(Instant);
impl tokio_timer::clock::Now for MockNow {
fn now(&self) -> Instant {
self.0
}
}
#[test]
fn clock_and_timer_concurrent() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = runtime::Builder::new()
.clock(clock)
.build()
.unwrap();
let (tx, rx) = mpsc::channel();
rt.spawn({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn clock_and_timer_single_threaded() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = current_thread::Builder::new()
.clock(clock)
.build()
.unwrap();
rt.block_on({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
Ok(())
})
}).unwrap();
}
-42
View File
@@ -1,42 +0,0 @@
extern crate tokio;
extern crate futures;
use std::thread;
use std::net;
use futures::future;
use futures::prelude::*;
use futures::sync::oneshot;
use tokio::net::TcpListener;
use tokio::reactor::Reactor;
#[test]
fn tcp_doesnt_block() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
drop(core);
assert!(listener.incoming().wait().next().unwrap().is_err());
}
#[test]
fn drop_wakes() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
let (tx, rx) = oneshot::channel::<()>();
let t = thread::spawn(move || {
let incoming = listener.incoming();
let new_socket = incoming.into_future().map_err(|_| ());
let drop_tx = future::lazy(|| {
drop(tx);
future::ok(())
});
assert!(new_socket.join(drop_tx).wait().is_err());
});
drop(rx.wait());
drop(core);
t.join().unwrap();
}
-27
View File
@@ -1,27 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_executor;
extern crate tokio_timer;
use futures::sync::mpsc;
use tokio::util::StreamExt;
#[test]
fn enumerate() {
use futures::*;
let (mut tx, rx) = mpsc::channel(1);
std::thread::spawn(|| {
for i in 0..5 {
tx = tx.send(i * 2).wait().unwrap();
}
});
let result = rx.enumerate().collect();
assert_eq!(
result.wait(),
Ok(vec![(0, 0), (1, 2), (2, 4), (3, 6), (4, 8)])
);
}
-136
View File
@@ -1,136 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn hammer_old() {
let _ = env_logger::try_init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().into_future()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(mine.join(theirs).wait());
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
}).collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
}
struct Rd(Arc<TcpStream>);
struct Wr(Arc<TcpStream>);
impl io::Read for Rd {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
<&TcpStream>::read(&mut &*self.0, dst)
}
}
impl tokio_io::AsyncRead for Rd {
}
impl io::Write for Wr {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
<&TcpStream>::write(&mut &*self.0, src)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl tokio_io::AsyncWrite for Wr {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
#[test]
fn hammer_split() {
use tokio_io::io;
const N: usize = 100;
const ITER: usize = 10;
let _ = env_logger::try_init();
for _ in 0..ITER {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let cnt = Arc::new(AtomicUsize::new(0));
let mut rt = Runtime::new().unwrap();
fn split(socket: TcpStream, cnt: Arc<AtomicUsize>) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let cnt2 = cnt.clone();
let rd = io::read(rd, vec![0; 1])
.map(move |_| {
cnt2.fetch_add(1, Relaxed);
})
.map_err(|e| panic!("read error = {:?}", e));
let wr = io::write_all(wr, b"1")
.map(move |_| {
cnt.fetch_add(1, Relaxed);
})
.map_err(move |e| panic!("write error = {:?}", e));
tokio::spawn(rd);
tokio::spawn(wr);
}
rt.spawn({
let cnt = cnt.clone();
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(move |socket| {
split(socket, cnt.clone());
Ok(())
})
});
for _ in 0..N {
rt.spawn({
let cnt = cnt.clone();
TcpStream::connect(&addr)
.map_err(move |e| panic!("connect error = {:?}", e))
.map(move |socket| split(socket, cnt))
});
}
rt.shutdown_on_idle().wait().unwrap();
assert_eq!(N * 4, cnt.load(Relaxed));
}
}
-580
View File
@@ -1,580 +0,0 @@
extern crate tokio;
extern crate futures;
extern crate bytes;
use tokio::io::{AsyncRead, AsyncWrite};
use tokio::codec::*;
use bytes::{Bytes, BytesMut, BufMut};
use futures::{Stream, Sink, Poll};
use futures::Async::*;
use std::io;
use std::collections::VecDeque;
macro_rules! mock {
($($x:expr,)*) => {{
let mut v = VecDeque::new();
v.extend(vec![$($x),*]);
Mock { calls: v }
}};
}
#[test]
fn read_empty_io_yields_nothing() {
let mut io = FramedRead::new(mock!(), LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet_little_endian() {
let mut io = length_delimited::Builder::new()
.little_endian()
.new_read(mock! {
Ok(b"\x09\x00\x00\x00abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet_native_endian() {
let data = if cfg!(target_endian = "big") {
b"\x00\x00\x00\x09abcdefghi"
} else {
b"\x09\x00\x00\x00abcdefghi"
};
let mut io = length_delimited::Builder::new()
.native_endian()
.new_read(mock! {
Ok(data[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"\x00\x00\x00\x09abcdefghi");
data.extend_from_slice(b"\x00\x00\x00\x03123");
data.extend_from_slice(b"\x00\x00\x00\x0bhello world");
let mut io = FramedRead::new(mock! {
Ok(data.into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_multi_packet() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Ok(b"\x00\x09abc"[..].into()),
Ok(b"defghi"[..].into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_multi_frame_multi_packet() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Ok(b"\x00\x09abc"[..].into()),
Ok(b"defghi"[..].into()),
Ok(b"\x00\x00\x00\x0312"[..].into()),
Ok(b"3\x00\x00\x00\x0bhello world"[..].into()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_multi_packet_wait() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09abc"[..].into()),
Err(would_block()),
Ok(b"defghi"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_multi_frame_multi_packet_wait() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09abc"[..].into()),
Err(would_block()),
Ok(b"defghi"[..].into()),
Err(would_block()),
Ok(b"\x00\x00\x00\x0312"[..].into()),
Err(would_block()),
Ok(b"3\x00\x00\x00\x0bhello world"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_incomplete_head() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00"[..].into()),
}, LengthDelimitedCodec::new());
assert!(io.poll().is_err());
}
#[test]
fn read_incomplete_head_multi() {
let mut io = FramedRead::new(mock! {
Err(would_block()),
Ok(b"\x00"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert!(io.poll().is_err());
}
#[test]
fn read_incomplete_payload() {
let mut io = FramedRead::new(mock! {
Ok(b"\x00\x00\x00\x09ab"[..].into()),
Err(would_block()),
Ok(b"cd"[..].into()),
Err(would_block()),
}, LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert!(io.poll().is_err());
}
#[test]
fn read_max_frame_len() {
let mut io = length_delimited::Builder::new()
.max_frame_length(5)
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_update_max_frame_len_at_rest() {
let mut io = length_delimited::Builder::new()
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
io.decoder_mut().set_max_frame_length(5);
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_update_max_frame_len_in_flight() {
let mut io = length_delimited::Builder::new()
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcd"[..].into()),
Err(would_block()),
Ok(b"efghi"[..].into()),
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), NotReady);
io.decoder_mut().set_max_frame_length(5);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_one_byte_length_field() {
let mut io = length_delimited::Builder::new()
.length_field_length(1)
.new_read(mock! {
Ok(b"\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_header_offset() {
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_field_offset(4)
.new_read(mock! {
Ok(b"zzzz\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet_skip_none_adjusted() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"xx\x00\x09abcdefghi");
data.extend_from_slice(b"yy\x00\x03123");
data.extend_from_slice(b"zz\x00\x0bhello world");
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_field_offset(2)
.num_skip(0)
.length_adjustment(4)
.new_read(mock! {
Ok(data.into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"xx\x00\x09abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"yy\x00\x03123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"zz\x00\x0bhello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet_length_includes_head() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"\x00\x0babcdefghi");
data.extend_from_slice(b"\x00\x05123");
data.extend_from_slice(b"\x00\x0dhello world");
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_adjustment(-2)
.new_read(mock! {
Ok(data.into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn write_single_frame_length_adjusted() {
let mut io = length_delimited::Builder::new()
.length_adjustment(-2)
.new_write(mock! {
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_nothing_yields_nothing() {
let mut io = FramedWrite::new(
mock!(),
LengthDelimitedCodec::new()
);
assert!(io.poll_complete().unwrap().is_ready());
}
#[test]
fn write_single_frame_one_packet() {
let mut io = FramedWrite::new(mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_multi_frame_one_packet() {
let mut io = FramedWrite::new(mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(b"\x00\x00\x00\x03"[..].into()),
Ok(b"123"[..].into()),
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"hello world"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.start_send(Bytes::from("123")).unwrap().is_ready());
assert!(io.start_send(Bytes::from("hello world")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_multi_frame_multi_packet() {
let mut io = FramedWrite::new(mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
Ok(b"\x00\x00\x00\x03"[..].into()),
Ok(b"123"[..].into()),
Ok(Flush),
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"hello world"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.start_send(Bytes::from("123")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.start_send(Bytes::from("hello world")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_would_block() {
let mut io = FramedWrite::new(mock! {
Err(would_block()),
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
}, LengthDelimitedCodec::new());
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_little_endian() {
let mut io = length_delimited::Builder::new()
.little_endian()
.new_write(mock! {
Ok(b"\x09\x00\x00\x00"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_with_short_length_field() {
let mut io = length_delimited::Builder::new()
.length_field_length(1)
.new_write(mock! {
Ok(b"\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_max_frame_len() {
let mut io = length_delimited::Builder::new()
.max_frame_length(5)
.new_write(mock! { });
assert_eq!(io.start_send(Bytes::from("abcdef")).unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_update_max_frame_len_at_rest() {
let mut io = length_delimited::Builder::new()
.new_write(mock! {
Ok(b"\x00\x00\x00\x06"[..].into()),
Ok(b"abcdef"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
io.encoder_mut().set_max_frame_length(5);
assert_eq!(io.start_send(Bytes::from("abcdef")).unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_update_max_frame_len_in_flight() {
let mut io = length_delimited::Builder::new()
.new_write(mock! {
Ok(b"\x00\x00\x00\x06"[..].into()),
Ok(b"ab"[..].into()),
Err(would_block()),
Ok(b"cdef"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
io.encoder_mut().set_max_frame_length(5);
assert!(io.poll_complete().unwrap().is_ready());
assert_eq!(io.start_send(Bytes::from("abcdef")).unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_zero() {
let mut io = length_delimited::Builder::new()
.new_write(mock! { });
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert_eq!(io.poll_complete().unwrap_err().kind(), io::ErrorKind::WriteZero);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn encode_overflow() {
// Test reproducing tokio-rs/tokio#681.
let mut codec = length_delimited::Builder::new().new_codec();
let mut buf = BytesMut::with_capacity(1024);
// Put some data into the buffer without resizing it to hold more.
let some_as = std::iter::repeat(b'a')
.take(1024)
.collect::<Vec<_>>();
buf.put_slice(&some_as[..]);
// Trying to encode the length header should resize the buffer if it won't fit.
codec.encode(Bytes::from("hello"), &mut buf).unwrap();
}
// ===== Test utils =====
fn would_block() -> io::Error {
io::Error::new(io::ErrorKind::WouldBlock, "would block")
}
struct Mock {
calls: VecDeque<io::Result<Op>>,
}
enum Op {
Data(Vec<u8>),
Flush,
}
use self::Op::*;
impl io::Read for Mock {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(Op::Data(data))) => {
debug_assert!(dst.len() >= data.len());
dst[..data.len()].copy_from_slice(&data[..]);
Ok(data.len())
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
}
impl AsyncRead for Mock {
}
impl io::Write for Mock {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(Op::Data(data))) => {
let len = data.len();
assert!(src.len() >= len, "expect={:?}; actual={:?}", data, src);
assert_eq!(&data[..], &src[..len]);
Ok(len)
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
fn flush(&mut self) -> io::Result<()> {
match self.calls.pop_front() {
Some(Ok(Op::Flush)) => {
Ok(())
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(()),
}
}
}
impl AsyncWrite for Mock {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(Ready(()))
}
}
impl<'a> From<&'a [u8]> for Op {
fn from(src: &'a [u8]) -> Op {
Op::Data(src.into())
}
}
impl From<Vec<u8>> for Op {
fn from(src: Vec<u8>) -> Op {
Op::Data(src)
}
}
-88
View File
@@ -1,88 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate tokio_threadpool;
extern crate bytes;
use std::io;
use std::net::Shutdown;
use bytes::{BytesMut, BufMut};
use futures::{Future, Stream, Sink};
use tokio::net::{TcpListener, TcpStream};
use tokio_codec::{Encoder, Decoder};
use tokio_io::io::{write_all, read};
use tokio_threadpool::Builder;
pub struct LineCodec;
impl Decoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<BytesMut>, io::Error> {
match buf.iter().position(|&b| b == b'\n') {
Some(i) => Ok(Some(buf.split_to(i + 1).into())),
None => Ok(None),
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() == 0 {
Ok(None)
} else {
let amt = buf.len();
Ok(Some(buf.split_to(amt)))
}
}
}
impl Encoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn encode(&mut self, item: BytesMut, into: &mut BytesMut) -> io::Result<()> {
into.put(&item[..]);
Ok(())
}
}
#[test]
fn echo() {
drop(env_logger::try_init());
let pool = Builder::new()
.pool_size(1)
.build();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
let addr = listener.local_addr().unwrap();
let sender = pool.sender().clone();
let srv = listener.incoming().for_each(move |socket| {
let (sink, stream) = LineCodec.framed(socket).split();
sender.spawn(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
Ok(())
});
pool.sender().spawn(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
let client = TcpStream::connect(&addr);
let client = client.wait().unwrap();
let (client, _) = write_all(client, b"a\n").wait().unwrap();
let (client, buf, amt) = read(client, vec![0; 1024]).wait().unwrap();
assert_eq!(amt, 2);
assert_eq!(&buf[..2], b"a\n");
let (client, _) = write_all(client, b"\n").wait().unwrap();
let (client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"\n");
let (client, _) = write_all(client, b"b").wait().unwrap();
client.shutdown(Shutdown::Write).unwrap();
let (_client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"b");
}
-88
View File
@@ -1,88 +0,0 @@
#![cfg(unix)]
extern crate env_logger;
extern crate futures;
extern crate libc;
extern crate mio;
extern crate tokio;
extern crate tokio_io;
use std::fs::File;
use std::io::{self, Write};
use std::os::unix::io::{AsRawFd, FromRawFd};
use std::thread;
use std::time::Duration;
use mio::event::Evented;
use mio::unix::{UnixReady, EventedFd};
use mio::{PollOpt, Ready, Token};
use tokio::reactor::{Handle, PollEvented2};
use tokio_io::io::read_to_end;
use futures::Future;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
struct MyFile(File);
impl MyFile {
fn new(file: File) -> MyFile {
unsafe {
let r = libc::fcntl(file.as_raw_fd(), libc::F_SETFL, libc::O_NONBLOCK);
assert!(r != -1, "fcntl error: {}", io::Error::last_os_error());
}
MyFile(file)
}
}
impl io::Read for MyFile {
fn read(&mut self, bytes: &mut [u8]) -> io::Result<usize> {
self.0.read(bytes)
}
}
impl Evented for MyFile {
fn register(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).register(poll, token, interest | hup, opts)
}
fn reregister(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).reregister(poll, token, interest | hup, opts)
}
fn deregister(&self, poll: &mio::Poll) -> io::Result<()> {
EventedFd(&self.0.as_raw_fd()).deregister(poll)
}
}
#[test]
fn hup() {
drop(env_logger::try_init());
let handle = Handle::default();
unsafe {
let mut pipes = [0; 2];
assert!(libc::pipe(pipes.as_mut_ptr()) != -1,
"pipe error: {}", io::Error::last_os_error());
let read = File::from_raw_fd(pipes[0]);
let mut write = File::from_raw_fd(pipes[1]);
let t = thread::spawn(move || {
write.write_all(b"Hello!\n").unwrap();
write.write_all(b"Good bye!\n").unwrap();
thread::sleep(Duration::from_millis(100));
});
let source = PollEvented2::new_with_handle(MyFile::new(read), &handle).unwrap();
let reader = read_to_end(source, Vec::new());
let (_, content) = t!(reader.wait());
assert_eq!(&b"Hello!\nGood bye!\n"[..], &content[..]);
t.join().unwrap();
}
}
-89
View File
@@ -1,89 +0,0 @@
extern crate futures;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_tcp;
use tokio_reactor::Reactor;
use tokio_tcp::TcpListener;
use futures::{Future, Stream};
use futures::executor::{spawn, Notify, Spawn};
use std::mem;
use std::net::TcpStream;
use std::sync::{Arc, Mutex};
#[test]
fn test_drop_on_notify() {
// When the reactor receives a kernel notification, it notifies the
// task that holds the associated socket. If this notification results in
// the task being dropped, the socket will also be dropped.
//
// Previously, there was a deadlock scenario where the reactor, while
// notifying, held a lock and the task being dropped attempted to acquire
// that same lock in order to clean up state.
//
// To simulate this case, we create a fake executor that does nothing when
// the task is notified. This simulates an executor in the process of
// shutting down. Then, when the task handle is dropped, the task itself is
// dropped.
struct MyNotify;
type Task = Mutex<Spawn<Box<Future<Item = (), Error = ()>>>>;
impl Notify for MyNotify {
fn notify(&self, _: usize) {
// Do nothing
}
fn clone_id(&self, id: usize) -> usize {
let ptr = id as *const Task;
let task = unsafe { Arc::from_raw(ptr) };
mem::forget(task.clone());
mem::forget(task);
id
}
fn drop_id(&self, id: usize) {
let ptr = id as *const Task;
let _ = unsafe { Arc::from_raw(ptr) };
}
}
let addr = "127.0.0.1:0".parse().unwrap();
let mut reactor = Reactor::new().unwrap();
// Create a listener
let listener = TcpListener::bind(&addr).unwrap();
let addr = listener.local_addr().unwrap();
// Define a task that just drains the listener
let task = Box::new({
listener.incoming()
.for_each(|_| Ok(()))
.map_err(|_| panic!())
}) as Box<Future<Item = (), Error = ()>>;
let task = Arc::new(Mutex::new(spawn(task)));
let notify = Arc::new(MyNotify);
let mut enter = tokio_executor::enter().unwrap();
tokio_reactor::with_default(&reactor.handle(), &mut enter, |_| {
let id = &*task as *const Task as usize;
task.lock().unwrap()
.poll_future_notify(&notify, id)
.unwrap();
});
drop(task);
// Establish a connection to the acceptor
let _s = TcpStream::connect(&addr).unwrap();
reactor.turn(None).unwrap();
}
-516
View File
@@ -1,516 +0,0 @@
extern crate tokio;
extern crate env_logger;
extern crate futures;
use futures::sync::oneshot;
use std::sync::{Arc, Mutex, atomic};
use std::thread;
use tokio::io;
use tokio::net::{TcpStream, TcpListener};
use tokio::prelude::future::lazy;
use tokio::prelude::*;
use tokio::runtime::Runtime;
// this import is used in all child modules that have it in scope
// from importing super::*, but the compiler doesn't realise that
// and warns about it.
pub use futures::future::Executor;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
fn create_client_server_future() -> Box<Future<Item=(), Error=()> + Send> {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| panic!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| ())
.map_err(|e| panic!("write err = {:?}", e))
})
})
.map(|_| ());
let client = client
.map_err(|e| panic!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| ())
.map_err(|e| panic!("read err = {:?}", e))
});
let future = server.join(client)
.map(|_| ());
Box::new(future)
}
#[test]
fn runtime_tokio_run() {
let _ = env_logger::try_init();
tokio::run(create_client_server_future());
}
#[test]
fn runtime_single_threaded() {
let _ = env_logger::try_init();
let mut runtime = tokio::runtime::current_thread::Runtime::new()
.unwrap();
runtime.block_on(create_client_server_future()).unwrap();
runtime.run().unwrap();
}
#[test]
fn runtime_single_threaded_block_on() {
let _ = env_logger::try_init();
tokio::runtime::current_thread::block_on_all(create_client_server_future()).unwrap();
}
mod runtime_single_threaded_block_on_all {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>),
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let msg = tokio::runtime::current_thread::block_on_all(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn spawn() {
test(|f| { tokio::spawn(f); })
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
}
mod runtime_single_threaded_racy {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(
tokio::runtime::current_thread::Handle,
Box<Future<Item=(), Error=()> + Send>,
),
{
let (trigger, exit) = futures::sync::oneshot::channel();
let (handle_tx, handle_rx) = ::std::sync::mpsc::channel();
let jh = ::std::thread::spawn(move || {
let mut rt = tokio::runtime::current_thread::Runtime::new().unwrap();
handle_tx.send(rt.handle()).unwrap();
// don't exit until we are told to
rt.block_on(exit.map_err(|_| ())).unwrap();
// run until all spawned futures (incl. the "exit" signal future) have completed.
rt.run().unwrap();
});
let (tx, rx) = futures::sync::oneshot::channel();
let handle = handle_rx.recv().unwrap();
spawn(handle, Box::new(futures::future::lazy(move || {
tx.send(()).unwrap();
Ok(())
})));
// signal runtime thread to exit
trigger.send(()).unwrap();
// wait for runtime thread to exit
jh.join().unwrap();
assert_eq!(rx.wait().unwrap(), ());
}
#[test]
fn spawn() {
test(|handle, f| { handle.spawn(f).unwrap(); })
}
#[test]
fn execute() {
test(|handle, f| { handle.execute(f).unwrap(); })
}
}
mod runtime_multi_threaded {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(&mut Runtime) + Send + 'static,
{
let _ = env_logger::try_init();
let mut runtime = tokio::runtime::Builder::new()
.build()
.unwrap();
spawn(&mut runtime);
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn() {
test(|rt| { rt.spawn(create_client_server_future()); });
}
#[test]
fn execute() {
test(|rt| { rt.executor().execute(create_client_server_future()).unwrap(); });
}
}
#[test]
fn block_on_timer() {
use std::time::{Duration, Instant};
use tokio::timer::{Delay, Error};
fn after_1s<T>(x: T) -> Box<Future<Item = T, Error = Error> + Send>
where
T: Send + 'static,
{
Box::new(Delay::new(Instant::now() + Duration::from_millis(100)).map(move |_| x))
}
let mut runtime = Runtime::new().unwrap();
assert_eq!(runtime.block_on(after_1s(42)).unwrap(), 42);
runtime.shutdown_on_idle().wait().unwrap();
}
mod from_block_on {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>) + Send + 'static,
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
let msg = runtime
.block_on(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
#[test]
fn spawn() {
test(|f| {
tokio::spawn(f);
})
}
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
use std::time::Duration;
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
runtime
.block_on(rx.then(move |_| {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<_, ()>(())
}))
.unwrap();
assert_eq!(1, *cnt.lock().unwrap());
runtime.shutdown_on_idle().wait().unwrap();
}
mod many {
use super::*;
const ITER: usize = 200;
fn test<F>(spawn: F)
where
F: Fn(&mut Runtime, Box<Future<Item=(), Error=()> + Send>),
{
let cnt = Arc::new(Mutex::new(0));
let mut runtime = Runtime::new().unwrap();
for _ in 0..ITER {
let c = cnt.clone();
spawn(&mut runtime, Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
}
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(ITER, *cnt.lock().unwrap());
}
#[test]
fn spawn() {
test(|rt, f| { rt.spawn(f); })
}
#[test]
fn execute() {
test(|rt, f| {
rt.executor()
.execute(f)
.unwrap();
})
}
}
mod from_block_on_all {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>) + Send + 'static,
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let runtime = Runtime::new().unwrap();
let msg = runtime
.block_on_all(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
#[test]
fn spawn() {
test(|f| { tokio::spawn(f); })
}
}
mod nested_enter {
use super::*;
use tokio::runtime::current_thread;
use std::panic;
fn test<F1, F2>(first: F1, nested: F2)
where
F1: Fn(Box<Future<Item=(), Error=()> + Send>) + Send + 'static,
F2: Fn(Box<Future<Item=(), Error=()> + Send>) + panic::UnwindSafe + Send + 'static,
{
let panicked = Arc::new(Mutex::new(false));
let panicked2 = panicked.clone();
// Since this is testing panics in other threads, printing about panics
// is noisy and can give the impression that the test is ignoring panics.
//
// It *is* ignoring them, but on purpose.
let prev_hook = panic::take_hook();
panic::set_hook(Box::new(|info| {
let s = info.to_string();
if s.starts_with("panicked at 'nested ")
|| s.starts_with("panicked at 'Multiple executors at once")
{
// expected, noop
} else {
println!("{}", s);
}
}));
first(Box::new(lazy(move || {
panic::catch_unwind(move || {
nested(Box::new(lazy(|| { Ok::<(), ()>(()) })))
}).expect_err("nested should panic");
*panicked2.lock().unwrap() = true;
Ok::<(), ()>(())
})));
panic::set_hook(prev_hook);
assert!(*panicked.lock().unwrap(), "nested call should have panicked");
}
fn threadpool_new() -> Runtime {
Runtime::new().expect("rt new")
}
#[test]
fn run_in_run() {
test(tokio::run, tokio::run);
}
#[test]
fn threadpool_block_on_in_run() {
test(tokio::run, |fut| {
let mut rt = threadpool_new();
rt.block_on(fut).unwrap();
});
}
#[test]
fn threadpool_block_on_all_in_run() {
test(tokio::run, |fut| {
let rt = threadpool_new();
rt.block_on_all(fut).unwrap();
});
}
#[test]
fn current_thread_block_on_all_in_run() {
test(tokio::run, |fut| {
current_thread::block_on_all(fut).unwrap();
});
}
}
#[test]
fn runtime_reactor_handle() {
#![allow(deprecated)]
use futures::Stream;
use std::net::{
TcpListener as StdListener,
TcpStream as StdStream,
};
let rt = Runtime::new().unwrap();
let std_listener = StdListener::bind("127.0.0.1:0").unwrap();
let tk_listener = TcpListener::from_std(std_listener, rt.handle()).unwrap();
let addr = tk_listener.local_addr().unwrap();
// Spawn a thread since we are avoiding the runtime
let th = thread::spawn(|| {
for _ in tk_listener.incoming().take(1).wait() {
}
});
let _ = StdStream::connect(&addr).unwrap();
th.join().unwrap();
}
#[test]
fn after_start_and_before_stop_is_called() {
let _ = env_logger::try_init();
let after_start = Arc::new(atomic::AtomicUsize::new(0));
let before_stop = Arc::new(atomic::AtomicUsize::new(0));
let after_inner = after_start.clone();
let before_inner = before_stop.clone();
let runtime = tokio::runtime::Builder::new()
.after_start(move || { after_inner.clone().fetch_add(1, atomic::Ordering::Relaxed); })
.before_stop(move || { before_inner.clone().fetch_add(1, atomic::Ordering::Relaxed); })
.build()
.unwrap();
runtime.block_on_all(create_client_server_future()).unwrap();
assert!(after_start.load(atomic::Ordering::Relaxed) > 0);
assert!(before_stop.load(atomic::Ordering::Relaxed) > 0);
}
-116
View File
@@ -1,116 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use tokio::prelude::*;
use tokio::timer::*;
use std::sync::mpsc;
use std::time::{Duration, Instant};
#[test]
fn timer_with_runtime() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(100);
let (tx, rx) = mpsc::channel();
tokio::run({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn starving() {
use futures::{task, Poll, Async};
let _ = env_logger::try_init();
struct Starve(Delay, u64);
impl Future for Starve {
type Item = u64;
type Error = ();
fn poll(&mut self) -> Poll<Self::Item, ()> {
if self.0.poll().unwrap().is_ready() {
return Ok(self.1.into());
}
self.1 += 1;
task::current().notify();
Ok(Async::NotReady)
}
}
let when = Instant::now() + Duration::from_millis(20);
let starve = Starve(Delay::new(when), 0);
let (tx, rx) = mpsc::channel();
tokio::run({
starve
.and_then(move |_ticks| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn deadline() {
use futures::future;
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(20);
let (tx, rx) = mpsc::channel();
#[allow(deprecated)]
tokio::run({
future::empty::<(), ()>()
.deadline(when)
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn timeout() {
use futures::future;
let _ = env_logger::try_init();
let (tx, rx) = mpsc::channel();
tokio::run({
future::empty::<(), ()>()
.timeout(Duration::from_millis(20))
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
-29
View File
@@ -1,29 +0,0 @@
[package]
name = "tokio-async-await"
# When releasing to crates.io:
# - Update html_root_url.
version = "0.1.5"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-async-await/0.1.3"
description = """
Experimental async/await support for Tokio
"""
categories = ["asynchronous"]
[features]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = ["futures/nightly"]
[dependencies]
futures = "0.1.23"
tokio-io = { version = "0.1.7", path = "../tokio-io" }
[dev-dependencies]
bytes = "0.4.9"
tokio = { version = "0.1.8", path = ".." }
hyper = "0.12.8"
-52
View File
@@ -1,52 +0,0 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
Copyright (c) 2016 futures-rs authors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
-55
View File
@@ -1,55 +0,0 @@
# Tokio async/await preview
This crate provides a preview of Tokio with async / await support. It is a shim
layer on top of `tokio`.
**This crate requires Rust nightly and does not provide API stability
guarantees. You are living on the edge here.**
## Usage
To use this crate, you need to start with a Rust 2018 edition crate, with rustc
1.33.0-nightly or later.
Add this to your `Cargo.toml`:
```toml
# In the `[packages]` section
edition = "2018"
# In the `[dependencies]` section
tokio = {version = "0.1.0", features = ["async-await-preview"]}
```
Then, get started. In your application, add:
```rust
// The nightly features that are commonly needed with async / await
#![feature(await_macro, async_await, futures_api)]
// This pulls in the `tokio-async-await` crate. While Rust 2018 doesn't require
// `extern crate`, we need to pull in the macros.
#[macro_use]
extern crate tokio;
fn main() {
// And we are async...
tokio::run_async(async {
println!("Hello");
});
}
```
Because nightly is required, run the app with `cargo +nightly run`
Check the [examples](examples) directory for more.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-2
View File
@@ -1,2 +0,0 @@
[build]
target-dir = "../../target"
-49
View File
@@ -1,49 +0,0 @@
[package]
name = "examples"
edition = "2018"
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
# Break out of the parent workspace
[workspace]
[[bin]]
name = "chat"
path = "src/chat.rs"
[[bin]]
name = "echo_client"
path = "src/echo_client.rs"
[[bin]]
name = "echo_server"
path = "src/echo_server.rs"
[[bin]]
name = "hyper"
path = "src/hyper.rs"
[dependencies]
tokio = { version = "0.1.0", path = "../..", features = ["async-await-preview"] }
futures = "0.1.23"
bytes = "0.4.9"
hyper = "0.12.8"
# Avoid using crates.io for Tokio dependencies
[patch.crates-io]
tokio = { path = "../.." }
tokio-async-await = { path = "../" }
tokio-codec = { path = "../../tokio-codec" }
tokio-current-thread = { path = "../../tokio-current-thread" }
tokio-executor = { path = "../../tokio-executor" }
tokio-fs = { path = "../../tokio-fs" }
tokio-io = { path = "../../tokio-io" }
tokio-reactor = { path = "../../tokio-reactor" }
tokio-signal = { path = "../../tokio-signal" }
tokio-tcp = { path = "../../tokio-tcp" }
tokio-threadpool = { path = "../../tokio-threadpool" }
tokio-timer = { path = "../../tokio-timer" }
tokio-tls = { path = "../../tokio-tls" }
tokio-udp = { path = "../../tokio-udp" }
tokio-uds = { path = "../../tokio-uds" }
-5
View File
@@ -1,5 +0,0 @@
# Tokio async/await examples
These are a separate crate in order to work around some cargo bugs. It also
allows `[patch]` to be used in `Cargo.toml` to ensure the correct lib versions
are being pulled in.
-135
View File
@@ -1,135 +0,0 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate futures; // v0.1
use tokio::codec::{LinesCodec, Decoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use std::collections::HashMap;
use std::io;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<String>;
struct Shared {
peers: HashMap<SocketAddr, Tx>,
}
impl Shared {
/// Create a new, empty, instance of `Shared`.
fn new() -> Self {
Shared {
peers: HashMap::new(),
}
}
}
async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()> {
let addr = stream.peer_addr().unwrap();
let mut lines = LinesCodec::new().framed(stream);
// Extract the peer's name
let name = match await!(lines.next()) {
Some(name) => name?,
None => {
// Disconnected early
return Ok(());
}
};
println!("`{}` is joining the chat", name);
let (tx, mut rx) = mpsc::unbounded();
// Register the socket
state.lock().unwrap()
.peers.insert(addr, tx);
// Split the `lines` handle into send and recv handles. This allows spawning
// separate tasks.
let (mut lines_tx, mut lines_rx) = lines.split();
// Spawn a task that receives all lines broadcasted to us from other peers
// and writes it to the client.
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line));
}
});
// Use the current task to read lines from the socket and broadcast them to
// other peers.
while let Some(message) = await!(lines_rx.next()) {
// TODO: Error handling
let message = message.unwrap();
let mut line = name.clone();
line.push_str(": ");
line.push_str(&message);
line.push_str("\r\n");
let state = state.lock().unwrap();
for (peer_addr, tx) in &state.peers {
if *peer_addr != addr {
// TODO: Error handling
tx.unbounded_send(line.clone()).unwrap();
}
}
}
// Remove the client from the shared state. Doing so will also result in the
// tx task to terminate.
state.lock().unwrap()
.peers.remove(&addr)
.expect("bug");
Ok(())
}
fn main() {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse().unwrap();
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
println!("server running on localhost:6142");
// Start the Tokio runtime.
tokio::run_async(async move {
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = match stream {
Ok(stream) => stream,
Err(_) => continue,
};
let state = state.clone();
tokio::spawn_async(async move {
if let Err(_) = await!(process(stream, state)) {
eprintln!("failed to process connection");
}
});
}
});
}
@@ -1,53 +0,0 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
use tokio::net::TcpStream;
use tokio::prelude::*;
use std::io;
use std::net::SocketAddr;
const MESSAGES: &[&str] = &[
"hello",
"world",
"one two three",
];
async fn run_client(addr: &SocketAddr) -> io::Result<()> {
let mut stream = await!(TcpStream::connect(addr))?;
// Buffer to read into
let mut buf = [0; 128];
for msg in MESSAGES {
println!(" > write = {:?}", msg);
// Write the message to the server
await!(stream.write_all_async(msg.as_bytes()))?;
// Read the message back from the server
await!(stream.read_exact_async(&mut buf[..msg.len()]))?;
assert_eq!(&buf[..msg.len()], msg.as_bytes());
}
Ok(())
}
fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Connect to the echo serveer
tokio::run_async(async move {
match await!(run_client(&addr)) {
Ok(_) => println!("done."),
Err(e) => eprintln!("echo client failed; error = {:?}", e),
}
});
}
@@ -1,45 +0,0 @@
#![feature(await_macro, async_await)]
#[macro_use]
extern crate tokio;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use std::net::SocketAddr;
fn handle(mut stream: TcpStream) {
tokio::spawn_async(async move {
let mut buf = [0; 1024];
loop {
match await!(stream.read_async(&mut buf)).unwrap() {
0 => break, // Socket closed
n => {
// Send the data back
await!(stream.write_all_async(&buf[0..n])).unwrap();
}
}
}
});
}
fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Bind the TCP listener
let listener = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
tokio::run_async(async {
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = stream.unwrap();
handle(stream);
}
});
}
-33
View File
@@ -1,33 +0,0 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate hyper;
use tokio::prelude::*;
use hyper::Client;
use std::time::Duration;
use std::str;
pub fn main() {
tokio::run_async(async {
let client = Client::new();
let uri = "http://httpbin.org/ip".parse().unwrap();
let response = await!({
client.get(uri)
.timeout(Duration::from_secs(10))
}).unwrap();
println!("Response: {}", response.status());
let mut body = response.into_body();
while let Some(chunk) = await!(body.next()) {
let chunk = chunk.unwrap();
println!("chunk = {}", str::from_utf8(&chunk[..]).unwrap());
}
});
}
-16
View File
@@ -1,16 +0,0 @@
/// Wait for a future to complete.
#[macro_export]
macro_rules! await {
($e:expr) => {{
use $crate::std_await;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
#[allow(unused_imports)]
use $crate::compat::backward::IntoAwaitable as IntoAwaitableBackward;
#[allow(unused_mut)]
let mut e = $e;
let e = e.into_awaitable();
std_await!(e)
}}
}
-89
View File
@@ -1,89 +0,0 @@
use futures::{Future, Poll};
use std::pin::Pin;
use std::future::{
Future as StdFuture,
};
use std::ptr::NonNull;
use std::task::{
LocalWaker,
Poll as StdPoll,
UnsafeWake,
Waker,
};
/// Convert an 0.3 `Future` to an 0.1 `Future`.
#[derive(Debug)]
pub struct Compat<T>(Pin<Box<T>>);
impl<T> Compat<T> {
/// Create a new `Compat` backed by `future`.
pub fn new(future: T) -> Compat<T> {
Compat(Box::pin(future))
}
}
/// Convert a value into one that can be used with `await!`.
pub trait IntoAwaitable {
type Awaitable;
fn into_awaitable(self) -> Self::Awaitable;
}
impl<T> IntoAwaitable for T
where T: StdFuture,
{
type Awaitable = Self;
fn into_awaitable(self) -> Self {
self
}
}
impl<T, Item, Error> Future for Compat<T>
where T: StdFuture<Output = Result<Item, Error>>,
{
type Item = Item;
type Error = Error;
fn poll(&mut self) -> Poll<Item, Error> {
use futures::Async::*;
let local_waker = noop_local_waker();
let res = self.0.as_mut().poll(&local_waker);
match res {
StdPoll::Ready(Ok(val)) => Ok(Ready(val)),
StdPoll::Ready(Err(err)) => Err(err),
StdPoll::Pending => Ok(NotReady),
}
}
}
// ===== NoopWaker =====
struct NoopWaker;
fn noop_local_waker() -> LocalWaker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { LocalWaker::new(w) }
}
fn noop_waker() -> Waker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { Waker::new(w) }
}
unsafe impl UnsafeWake for NoopWaker {
unsafe fn clone_raw(&self) -> Waker {
noop_waker()
}
unsafe fn drop_raw(&self) {
}
unsafe fn wake(&self) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
}
-68
View File
@@ -1,68 +0,0 @@
use futures::{Future, Async};
use std::marker::Unpin;
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::task::{LocalWaker, Poll as StdPoll};
/// Converts an 0.1 `Future` into an 0.3 `Future`.
#[derive(Debug)]
pub struct Compat<T>(T);
pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T, E>> {
use futures::Async::{Ready, NotReady};
match poll {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
Ok(NotReady) => StdPoll::Pending,
Err(err) => StdPoll::Ready(Err(err)),
}
}
pub(crate) fn convert_poll_stream<T, E>(
poll: Result<Async<Option<T>>, E>) -> StdPoll<Option<Result<T, E>>>
{
use futures::Async::{Ready, NotReady};
match poll {
Ok(Ready(Some(val))) => StdPoll::Ready(Some(Ok(val))),
Ok(Ready(None)) => StdPoll::Ready(None),
Ok(NotReady) => StdPoll::Pending,
Err(err) => StdPoll::Ready(Some(Err(err))),
}
}
/// Convert a value into one that can be used with `await!`.
pub trait IntoAwaitable {
type Awaitable;
/// Convert `self` into a value that can be used with `await!`.
fn into_awaitable(self) -> Self::Awaitable;
}
impl<T: Future + Unpin> IntoAwaitable for T {
type Awaitable = Compat<T>;
fn into_awaitable(self) -> Self::Awaitable {
Compat(self)
}
}
impl<T> StdFuture for Compat<T>
where T: Future + Unpin
{
type Output = Result<T::Item, T::Error>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> StdPoll<Self::Output> {
use futures::Async::{Ready, NotReady};
// TODO: wire in cx
match self.0.poll() {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
Ok(NotReady) => StdPoll::Pending,
Err(e) => StdPoll::Ready(Err(e)),
}
}
}
-4
View File
@@ -1,4 +0,0 @@
#![doc(hidden)]
pub mod forward;
pub mod backward;
-32
View File
@@ -1,32 +0,0 @@
use tokio_io::AsyncWrite;
use std::io;
use std::future::Future;
use std::marker::Unpin;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
/// A future used to fully flush an I/O object.
#[derive(Debug)]
pub struct Flush<'a, T: ?Sized + 'a> {
writer: &'a mut T,
}
// Pin is never projected to fields
impl<'a, T: ?Sized> Unpin for Flush<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> Flush<'a, T> {
pub(super) fn new(writer: &'a mut T) -> Flush<'a, T> {
Flush { writer }
}
}
impl<'a, T: AsyncWrite + ?Sized> Future for Flush<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _wx: &LocalWaker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
convert_poll(self.writer.poll_flush())
}
}
-192
View File
@@ -1,192 +0,0 @@
//! Use I/O with `async` / `await`.
mod flush;
mod read;
mod read_exact;
mod write;
mod write_all;
pub use self::flush::Flush;
pub use self::read::Read;
pub use self::read_exact::ReadExact;
pub use self::write::Write;
pub use self::write_all::WriteAll;
use tokio_io::{AsyncRead, AsyncWrite};
/// An extension trait which adds utility methods to `AsyncRead` types.
pub trait AsyncReadExt: AsyncRead {
/// Tries to read some bytes directly into the given `buf` in an
/// asynchronous manner, returning a future.
///
/// The returned future will resolve to the number of bytes read once the read
/// operation is completed.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncReadExt;
/// use std::io::Cursor;
///
/// let mut reader = Cursor::new([1, 2, 3, 4]);
/// let mut output = [0u8; 5];
///
/// let bytes = await!(reader.read_async(&mut output[..])).unwrap();
///
/// // This is only guaranteed to be 4 because `&[u8]` is a synchronous
/// // reader. In a real system you could get anywhere from 1 to
/// // `output.len()` bytes in a single read.
/// assert_eq!(bytes, 4);
/// assert_eq!(output, [1, 2, 3, 4, 0]);
/// });
/// ```
fn read_async<'a>(&'a mut self, buf: &'a mut [u8]) -> Read<'a, Self> {
Read::new(self, buf)
}
/// Creates a future which will read exactly enough bytes to fill `buf`,
/// returning an error if end of file (EOF) is hit sooner.
///
/// The returned future will resolve once the read operation is completed.
///
/// In the case of an error the buffer and the object will be discarded, with
/// the error yielded.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncReadExt;
/// use std::io::Cursor;
///
/// let mut reader = Cursor::new([1, 2, 3, 4]);
/// let mut output = [0u8; 4];
///
/// await!(reader.read_exact_async(&mut output)).unwrap();
///
/// assert_eq!(output, [1, 2, 3, 4]);
/// });
/// ```
///
/// ## EOF is hit before `buf` is filled
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncReadExt;
/// use std::io::{self, Cursor};
///
/// let mut reader = Cursor::new([1, 2, 3, 4]);
/// let mut output = [0u8; 5];
///
/// let result = await!(reader.read_exact_async(&mut output));
///
/// assert_eq!(result.unwrap_err().kind(), io::ErrorKind::UnexpectedEof);
/// });
/// ```
fn read_exact_async<'a>(&'a mut self, buf: &'a mut [u8]) -> ReadExact<'a, Self> {
ReadExact::new(self, buf)
}
}
/// An extension trait which adds utility methods to `AsyncWrite` types.
pub trait AsyncWriteExt: AsyncWrite {
/// Write data into this object.
///
/// Creates a future that will write the entire contents of the buffer `buf` into
/// this `AsyncWrite`.
///
/// The returned future will not complete until all the data has been written.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncWriteExt;
/// use std::io::Cursor;
///
/// let mut buf = [0u8; 5];
/// let mut writer = Cursor::new(&mut buf[..]);
///
/// let n = await!(writer.write_async(&[1, 2, 3, 4])).unwrap();
///
/// assert_eq!(writer.into_inner()[..n], [1, 2, 3, 4, 0][..n]);
/// });
/// ```
fn write_async<'a>(&'a mut self, buf: &'a [u8]) -> Write<'a, Self> {
Write::new(self, buf)
}
/// Write an entire buffer into this object.
///
/// Creates a future that will write the entire contents of the buffer `buf` into
/// this `AsyncWrite`.
///
/// The returned future will not complete until all the data has been written.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncWriteExt;
/// use std::io::Cursor;
///
/// let mut buf = [0u8; 5];
/// let mut writer = Cursor::new(&mut buf[..]);
///
/// await!(writer.write_all_async(&[1, 2, 3, 4])).unwrap();
///
/// assert_eq!(writer.into_inner(), [1, 2, 3, 4, 0]);
/// });
/// ```
fn write_all_async<'a>(&'a mut self, buf: &'a [u8]) -> WriteAll<'a, Self> {
WriteAll::new(self, buf)
}
/// Creates a future which will entirely flush this `AsyncWrite`.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncWriteExt;
/// use std::io::{BufWriter, Cursor};
///
/// let mut output = [0u8; 5];
///
/// {
/// let mut writer = Cursor::new(&mut output[..]);
/// let mut buffered = BufWriter::new(writer);
/// await!(buffered.write_all_async(&[1, 2])).unwrap();
/// await!(buffered.write_all_async(&[3, 4])).unwrap();
/// await!(buffered.flush_async()).unwrap();
/// }
///
/// assert_eq!(output, [1, 2, 3, 4, 0]);
/// });
/// ```
fn flush_async<'a>(&mut self) -> Flush<Self> {
Flush::new(self)
}
}
impl<T: AsyncRead + ?Sized> AsyncReadExt for T {}
impl<T: AsyncWrite + ?Sized> AsyncWriteExt for T {}
-38
View File
@@ -1,38 +0,0 @@
use tokio_io::AsyncRead;
use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future which can be used to read bytes.
#[derive(Debug)]
pub struct Read<'a, T: ?Sized + 'a> {
reader: &'a mut T,
buf: &'a mut [u8],
}
// Pinning is never projected to fields
impl<'a, T: ?Sized> Unpin for Read<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> Read<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> Read<'a, T> {
Read {
reader,
buf,
}
}
}
impl<'a, T: AsyncRead + ?Sized> Future for Read<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
convert_poll(this.reader.poll_read(this.buf))
}
}
-56
View File
@@ -1,56 +0,0 @@
use tokio_io::AsyncRead;
use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
/// A future which can be used to read exactly enough bytes to fill a buffer.
#[derive(Debug)]
pub struct ReadExact<'a, T: ?Sized + 'a> {
reader: &'a mut T,
buf: &'a mut [u8],
}
// Pinning is never projected to fields
impl<'a, T: ?Sized> Unpin for ReadExact<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> ReadExact<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> ReadExact<'a, T> {
ReadExact {
reader,
buf,
}
}
}
fn eof() -> io::Error {
io::Error::new(io::ErrorKind::UnexpectedEof, "early eof")
}
impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
while !this.buf.is_empty() {
let n = try_ready!(convert_poll(this.reader.poll_read(this.buf)));
{
let (_, rest) = mem::replace(&mut this.buf, &mut []).split_at_mut(n);
this.buf = rest;
}
if n == 0 {
return Poll::Ready(Err(eof()))
}
}
Poll::Ready(Ok(()))
}
}
-38
View File
@@ -1,38 +0,0 @@
use tokio_io::AsyncWrite;
use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future used to write data.
#[derive(Debug)]
pub struct Write<'a, T: 'a + ?Sized> {
writer: &'a mut T,
buf: &'a [u8],
}
// Pinning is never projected to fields
impl<'a, T: ?Sized> Unpin for Write<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> Write<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> Write<'a, T> {
Write {
writer,
buf,
}
}
}
impl<'a, T: AsyncWrite + ?Sized> Future for Write<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<usize>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
convert_poll(this.writer.poll_write(this.buf))
}
}
-57
View File
@@ -1,57 +0,0 @@
use tokio_io::AsyncWrite;
use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
/// A future used to write the entire contents of a buffer.
#[derive(Debug)]
pub struct WriteAll<'a, T: ?Sized + 'a> {
writer: &'a mut T,
buf: &'a [u8],
}
// Pinning is never projected to fields
impl<'a, T: ?Sized> Unpin for WriteAll<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> WriteAll<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> WriteAll<'a, T> {
WriteAll {
writer,
buf,
}
}
}
fn zero_write() -> io::Error {
io::Error::new(io::ErrorKind::WriteZero, "zero-length write")
}
impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<()>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
while !this.buf.is_empty() {
let n = try_ready!(convert_poll(this.writer.poll_write(this.buf)));
{
let (_, rest) = mem::replace(&mut this.buf, &[]).split_at(n);
this.buf = rest;
}
if n == 0 {
return Poll::Ready(Err(zero_write()))
}
}
Poll::Ready(Ok(()))
}
}
-114
View File
@@ -1,114 +0,0 @@
#![cfg(feature = "async-await-preview")]
#![feature(
rust_2018_preview,
arbitrary_self_types,
async_await,
await_macro,
futures_api,
)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.5")]
#![deny(missing_docs, missing_debug_implementations)]
#![cfg_attr(test, deny(warnings))]
//! A preview of Tokio w/ `async` / `await` support.
extern crate futures;
extern crate tokio_io;
/// Extracts the successful type of a `Poll<Result<T, E>>`.
///
/// This macro bakes in propagation of `Pending` and `Err` signals by returning early.
macro_rules! try_ready {
($x:expr) => {
match $x {
std::task::Poll::Ready(Ok(x)) => x,
std::task::Poll::Ready(Err(e)) =>
return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending =>
return std::task::Poll::Pending,
}
}
}
#[macro_use]
mod await;
pub mod compat;
pub mod io;
pub mod sink;
pub mod stream;
/*
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_main::prelude::*;
#[doc(inline)]
pub use crate::async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt,
},
};
}
*/
// Rename the `await` macro in `std`. This is used by the redefined
// `await` macro in this crate.
#[doc(hidden)]
pub use std::await as std_await;
/*
use std::future::{Future as StdFuture};
fn run<T: futures::Future<Item = (), Error = ()>>(t: T) {
drop(t);
}
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
run(future);
unimplemented!();
}
*/
/*
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use crate::async_await::compat::backward;
spawn(backward::Compat::new(async || {
let _ = await!(future);
Ok(())
}));
}
*/
-26
View File
@@ -1,26 +0,0 @@
//! Use sinks with `async` / `await`.
mod send;
pub use self::send::Send;
use futures::Sink;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Sink` types.
pub trait SinkExt: Sink {
/// Send an item into the sink.
///
/// Note that, **because of the flushing requirement, it is usually better
/// to batch together items to send via `send_all`, rather than flushing
/// between each item.**
fn send_async(&mut self, item: Self::SinkItem) -> Send<Self>
where
Self: Sized + Unpin,
{
Send::new(self, item)
}
}
impl<T: Sink> SinkExt for T {}
-54
View File
@@ -1,54 +0,0 @@
use futures::Sink;
use std::future::Future;
use std::task::{self, Poll};
use std::marker::Unpin;
use std::pin::Pin;
/// Future for the `SinkExt::send_async` combinator, which sends a value to a
/// sink and then waits until the sink has fully flushed.
#[derive(Debug)]
pub struct Send<'a, T: Sink + 'a + ?Sized> {
sink: &'a mut T,
item: Option<T::SinkItem>,
}
impl<T: Sink + Unpin + ?Sized> Unpin for Send<'_, T> {}
impl<'a, T: Sink + Unpin + ?Sized> Send<'a, T> {
pub(super) fn new(sink: &'a mut T, item: T::SinkItem) -> Self {
Send {
sink,
item: Some(item),
}
}
}
impl<T: Sink + Unpin + ?Sized> Future for Send<'_, T> {
type Output = Result<(), T::SinkError>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
use futures::AsyncSink::{Ready, NotReady};
if let Some(item) = self.item.take() {
match self.sink.start_send(item) {
Ok(Ready) => {}
Ok(NotReady(val)) => {
self.item = Some(val);
return Poll::Pending;
}
Err(err) => {
return Poll::Ready(Err(err));
}
}
}
// we're done sending the item, but want to block on flushing the
// sink
try_ready!(convert_poll(self.sink.poll_complete()));
Poll::Ready(Ok(()))
}
}
-40
View File
@@ -1,40 +0,0 @@
//! Use streams with `async` / `await`.
mod next;
pub use self::next::Next;
use futures::Stream;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Stream` types.
pub trait StreamExt: Stream {
/// Creates a future that resolves to the next item in the stream.
///
/// # Examples
///
/// ```
/// #![feature(await_macro, async_await)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::{stream, StreamExt};
///
/// let mut stream = stream::iter_ok::<_, ()>(1..3);
///
/// assert_eq!(await!(stream.next()), Some(Ok(1)));
/// assert_eq!(await!(stream.next()), Some(Ok(2)));
/// assert_eq!(await!(stream.next()), Some(Ok(3)));
/// assert_eq!(await!(stream.next()), None);
/// });
/// ```
fn next(&mut self) -> Next<Self>
where
Self: Sized + Unpin,
{
Next::new(self)
}
}
impl<T: Stream> StreamExt for T {}
-30
View File
@@ -1,30 +0,0 @@
use futures::Stream;
use std::future::Future;
use std::marker::Unpin;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
/// A future of the next element of a stream.
#[derive(Debug)]
pub struct Next<'a, T: 'a> {
stream: &'a mut T,
}
impl<'a, T: Stream + Unpin> Unpin for Next<'a, T> {}
impl<'a, T: Stream + Unpin> Next<'a, T> {
pub(super) fn new(stream: &'a mut T) -> Next<'a, T> {
Next { stream }
}
}
impl<'a, T: Stream + Unpin> Future for Next<'a, T> {
type Output = Option<Result<T::Item, T::Error>>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll_stream;
convert_poll_stream(self.stream.poll())
}
}
-3
View File
@@ -1,3 +0,0 @@
# 0.1.0 (unreleased)
* Initial release

Some files were not shown because too many files have changed in this diff Show More