Compare commits

...
Author SHA1 Message Date
Sean McArthur 15dc0563b7 prepare v0.2.0-alpha.4 (#1509) 2019-08-29 12:59:10 -07:00
Sean McArthur 4e26258ac3 Re-add temporarily TcpStream::connect_std (#1508) 2019-08-29 11:45:17 -07:00
Carl Lerche a59e096c47 prepare v0.2.0-alpha.3 release (#1505) 2019-08-28 15:04:42 -07:00
Carl Lerche fc1640891e net: perform DNS lookup on connect / bind. (#1499)
A sealed `net::ToSocketAddrs` trait is added. This trait is not intended
to be used by users. Instead, it is an argument to `connect` and `bind`
functions.

The operating system's DNS lookup functionality is used. Blocking
operations are performed on a thread pool in order to avoid blocking the
runtime.
2019-08-28 13:25:50 -07:00
Jakub Beránek de9f05d4d3 docs: fix wording in tokio_process::Child documentation (#1502)
Fixes: #1494
2019-08-28 11:56:42 -04:00
Eliza Weisman 9c31797a08 net: switch from log to tracing (#1455)
* net: switch from `log` to `tracing`.

Motivation:

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime.

Solution:

This branch replaces the use of `log` in `tokio-net` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation.

Notes:

I removed the timing in `Reactor::poll` in favor of simply adding a
`#[tracing::instrument]` attribute. Since the generated `tracing` span
will have enter and exit events, a `tracing::Subscriber`
implemementation can use those to record timestamps, and process that
timing data in a much more sophisticated manner than including it in a
log line.

We can add the timestamps back if they're desired.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-27 17:53:57 -07:00
Ömer Sinan Ağacan d1c58b7940 tokio: export RunError in tokio::runtime::current_thread (#1487)
This type is used in return type of `Runtime::run`, but because the type
was not exported it was opaque in the documentation of `Runtime`.
2019-08-27 12:26:48 -07:00
Jacob Pratt 5f74a99ea3 implement spawn_with_handle in tokio_executor (#1492)
This code directly relies on `future-preview`'s `RemoteHandle`, and
exposes it via a `spawn_with_handle` method that is identical to
`future-preview`'s implementation.

Related: #1180
2019-08-27 12:26:11 -07:00
Carl Lerche 08e20fcf6a fs: add support for non-threadpool executors (#1495)
Provides a thread pool dedicated to running blocking operations (#588)
and update `tokio-fs` to use this pool.

In an effort to make incremental progress, this is an initial step
towards a final solution. First, it provides a very basic pool
implementation with the intend that the pool will be
replaced before the final release. Second, it updates `tokio-fs` to
always use this blocking pool instead of conditionally using
`threadpool::blocking`. Issue #588 contains additional discussion around
potential improvements to the "blocking for all" strategy.

The implementation provided here builds on work started in #954 and
continued in #1045. The general idea is th same as #1045, but the PR
improves on some of the details:

* The number of explicit operations tracked by `File` is reduced only to
  the ones that could interact. All other ops are spawned on the
  blocking pool without being tracked by the `File` instance.

* The `seek` implementation is not backed by a trait and `poll_seek`
  function. This avoids the question of how to model non-blocking seeks
  on top of a blocking file. In this patch, `seek` is represented as an
  `async fn`. If the associated future is dropped before the caller
  observes the return value, we make no effort to define the state in
  which the file ends up.
2019-08-27 12:25:20 -07:00
Carl Lerche 08099bb2d3 net: rewrite TcpStream::connect with async fn (#1497)
This also removes `TcpStream::connect_std` as the conversion functions
from `std` need to be rethought. A note tracking this has been added
to #1209.
2019-08-27 12:08:28 -07:00
Newton Ni 807d536846 codec: fix infinite loop in tokio_codec::LinesCodec (#1489) 2019-08-26 13:38:52 -07:00
Danny Browning 654f9d703f tokio: expose signal feature (#1491)
Expose tokio_net::signal::ctrl_c via tokio::net::signal::ctrl_c as feature signal.
2019-08-22 09:12:00 -07:00
Jon Gjengset a285689664 net: shutdown TCP write when asked to shut down (#1488) 2019-08-21 10:41:51 -07:00
Gurwinder Singh 13930eff2a chore: two async_await feature remained (#1486) 2019-08-21 18:53:42 +09:00
Taiki Endo 24fb33e012 io: add AsyncReadExt::{chain, take} (#1484) 2019-08-20 20:09:07 -07:00
Taiki Endo a791f4a758 chore: bump to newer nightly (#1485) 2019-08-20 20:07:16 -07:00
Eliza Weisman 7e7a5147a3 executor: switch from log to tracing (#1454)
## Motivation

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime. Updating the executor crate is an obvious
starting point. 

## Solution

This branch replaces the use of `log` in `tokio-executor` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation. I've also added a few `tracing` spans, primarily in
places where a variable is added to all the log messages in a scope.

## Notes

For users who are using the legacy `log` output, there is a feature flag
to enable `log` support in `tracing`. I thought about making this on by
default, but that would also enable the `tracing` dependency by default,
and it is only pulled in when the `threadpool` feature flag is enabled.
The `tokio` crate could enable the log feature in its default features
instead, since the threadpool feature is on by default in `tokio`. If
this isn't the right approach, I can change how `log` back-compatibility
is enabled.

We might want to consider adding more `tracing` spans in the threadpool
later. This could be useful for profiling, and for helping users debug
the way their applications interact with the executor. This branch is
just intended as a starting point so that we can begin emitting
`tracing` data from the executor; we should revisit what instrumentation
should be exposed, as well.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-20 12:44:26 -07:00
Jakub Beránek 2d56312b89 timer: introduce delay function shortcut (#1440)
This commit adds a simple delay shortcut to avoid writing Delay::new
everywhere and removes usages of Delay::new.
2019-08-20 08:39:55 -07:00
Ivan Petkov 357df38861 process: move into the tokio-net crate (#1475) 2019-08-19 19:42:54 -07:00
John-John Tedro 34a9dc2d76 Implement FusedStream and FusedFuture for Interval and Delay (#1476) 2019-08-19 10:21:34 -04:00
Ivan Petkov 68d5fcb8d1 docs: fix all rustdoc warnings (#1474) 2019-08-18 14:38:54 -07:00
Ivan Petkov 08b07afbd9 signal: remove new() constructors in favor of free functions (#1472)
* Also removed any `*_with_handle` related methods in favor of always
using the default reactor
2019-08-18 14:22:09 -07:00
Douman 7b0c60849c net: make default reactor guard public (#1468) 2019-08-18 11:36:21 -07:00
Ivan Petkov 6d8d388dc5 docs: add docs.rs metadata to build with all features (#1471) 2019-08-18 11:11:46 -07:00
Ivan Petkov bc61bd9d3d ci: ensure all tests are run for each feature (#1470)
* This includes running docs, examples, and lib tests for each added
feature, to ensure nothing is broken
2019-08-18 10:50:38 -07:00
Jakub Beránek a9585f0318 tokio-process: change CommandExt to a fully asynchronous Command struct (#1448)
Refs: #1371
2019-08-18 10:13:37 -07:00
Carl Lerche 88b4ec84d7 chore: prepare 0.2.0-alpha.2 release (#1465) 2019-08-17 23:34:25 -07:00
Philip Kannegaard Hayes 9f0daad5ac sync: fix fuzz_oneshot test by using instrumented loom::sync::Arc (#1464)
Since `tokio_sync::oneshot` makes a `CausalCell::with_mut()` mutable
access in the `Inner::drop()`, we must use the instrumented
`loom::sync::Arc`.

Uncovered by carllerche/loom#42
2019-08-17 21:30:31 -07:00
Carl Lerche c187cd75b6 signal: move into tokio-net (#1463) 2019-08-17 13:43:55 -07:00
Carl Lerche a83f5e4ba6 uds: move into tokio-net (#1462) 2019-08-16 14:42:05 -07:00
Carl Lerche 4935aae164 udp: remove files left over from moving tokio-udp (#1461) 2019-08-16 11:05:50 -07:00
Carl Lerche ba1829fd26 chore: rename ui-tests -> build-tests (#1460) 2019-08-16 09:26:56 -07:00
Carl Lerche ce7e60e396 udp: move tokio-udp into tokio-net (#1459) 2019-08-16 07:26:10 -07:00
Ivan Petkov d8b23ef852 signal: rename SignalKind methods (#1457)
This renames the SignalKind constructors to be a bit more readable
instead of using the signal names themselves
2019-08-15 21:09:09 -07:00
Carl Lerche 4788d3a9e3 tcp: move tokio-tcp into tokio-net (#1456) 2019-08-15 20:37:25 -07:00
Carl Lerche f1f61a3b15 net: reorganize crate in anticipation of #1264 (#1453)
Space is made to add `tcp`, `udp`, `uds`, ... modules.
2019-08-15 15:04:21 -07:00
Jakub Beránek d0a8e5d6f2 tokio-fs: rewrite std echo example using async/await (#1442)
This PR fixes the echo example in tokio-fs.

Refs: #1255
2019-08-15 13:10:17 -07:00
Carl Lerche 3b27dc31d2 threadpool: move threadpool into tokio-executor (#1452)
The threadpool is behind a feature flag.

Refs: #1264
2019-08-15 13:09:02 -07:00
Douman 37131b2114 runtime: refactor thread-local setters (#1449) 2019-08-15 13:00:57 -07:00
Carl Lerche 8538c25170 reactor: rename tokio-reactor -> tokio-net (#1450)
* reactor: rename tokio-reactor -> tokio-net

This is in preparation for #1264
2019-08-15 11:04:58 -07:00
Jakub Beránek 7b6438a172 tokio: rewrite print_each_packet example using async/await (#1446)
This PR fixes the print each packet example in tokio.

Refs: #1201
2019-08-15 10:42:34 -07:00
Carl Lerche 9de7083be8 executor: move current-thread into crate (#1447)
The `CurrentThread` executor is exposed using a feature flag.

Refs: #1264
2019-08-15 09:52:25 -07:00
John Doneth 8d55f98f6f udp: update tokio_udp::UdpFramed to std::future (#1370) 2019-08-14 11:18:21 -07:00
Taiki Endo 999a600494 io: add async BufReader/BufWriter (#1438) 2019-08-14 10:24:07 -07:00
Ilya Lakhin fb9809c068 executor, threadpool: forward port fix from #1155 (#1433)
Add executor::exit, allowing other executors inside threadpool::blocking.
2019-08-13 21:12:49 -07:00
Douman 517162792f macros: upgrade syn/quote (#1432) 2019-08-13 21:11:26 -07:00
Geoff Shannon fe90d61446 test: add a block_on function to tokio-test (#1431) 2019-08-13 21:10:26 -07:00
Ivan Petkov 338b37884a signal: Add SignalKind for registering signals more easily (#1430)
This avoids having consumers import libc for common signals, and it
improves discoverability since users need not be aware that libc
contains all supported constants.
2019-08-13 21:07:22 -07:00
Ivan Petkov 513326e01d signal: remove driver task for Windows event implementation (#1429)
Windows guarantees handler routines are always invoked in a new thread
(https://docs.microsoft.com/en-us/windows/console/handlerroutine), so we
don't need to use the handler-wake-another-driver technique used in the
Unix implementation

By broadcasting the event notifications from the handler, we no longer
need the Driver task to be spawned, which fixes the starvation issue if
the executor which runs the Driver task goes away

Also changed the behavior so that the default event handler runs if
all listeners for CTRL_{C, BREAK} events go away.
2019-08-13 21:01:06 -07:00
Ivan Petkov 73a91ad7b3 signal: delete blocking Read/Write impls on ChildStd{in, out, err} (#1428) 2019-08-13 20:53:02 -07:00
Taiki Endo 930cce8677 chore: update futures-preview to 0.3.0-alpha.18 (#1427) 2019-08-10 14:09:28 -07:00
Taiki Endo 6a125082e4 chore: apply unreachable_pub and missing_debug_implementations to all crates (#1424) 2019-08-11 04:28:52 +09:00
Taiki Endo d9f9c5658f chore: bump to newer nightly (#1426) 2019-08-11 02:01:20 +09:00
Taiki Endo fff39c03b1 ci: deny warnings in cirrus (#1425) 2019-08-11 01:41:51 +09:00
Tomasz Miąsko 756606a58b uds: implement split and split_mut for UnixStream (#1395)
This mirrors split API available in TcpStream.
2019-08-09 12:50:18 -07:00
Ran Benita e3b4c99a33 codec: a few suggestions (#1418)
How the buffer is managed is often critical for performance. Not
taking care of it will be catastrophic for performance beyond the
initial buffer size with the current implementation (a loop of
`reserve(1)`).
2019-08-09 12:20:31 -07:00
Taiki Endo 42fa0c28d3 timer: use std::sync::atomic::AtomicU64 instead of own AtomicU64 (#1421) 2019-08-10 03:42:03 +09:00
Taiki Endo f7b41c9dcc macros: improve error messages (#1420) 2019-08-09 10:28:22 -07:00
Taiki Endo 73102760cf chore: change default lint level to warning and deny warnings in CI (#1416) 2019-08-10 00:07:57 +09:00
Douman 18833a8e67 macros: Error on function with arguments (#1419) 2019-08-09 11:04:41 -04:00
tmiasko eba8bf2b4b io: implement AsyncWrite for Vec<u8> (#1409) 2019-08-08 20:55:27 -07:00
David Kellum 790d649dc5 update (dev dep) env_logger to latest 0.6 (#1390) 2019-08-08 20:37:32 -07:00
Lucio Franco 50e5d401df chore: prepare for v0.2.0-alpha.1 release (#1410) 2019-08-08 12:48:53 -07:00
Carl Lerche 2e69f2a7fd sync: track upstream loom changes (#1407) 2019-08-07 23:24:22 -07:00
Carl Lerche 962521f449 chore: enable full CI run (#1399)
* update all tests
* fix doc examples
* misc API tweaks
2019-08-07 20:02:13 -07:00
Carl Lerche 831be9c08e executor: remove unused dependency (#1406) 2019-08-07 19:55:42 -07:00
Carl Lerche 23c380a78f sync: track loom changes (#1405) 2019-08-07 15:38:34 -07:00
Lucio Franco 0a05332648 Remove git dep and add macro examples (#1404)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 15:02:38 -07:00
Lucio Franco 7268b0bb3a Migrate threadpool to futures-util (#1403)
* Migrate threadpool to futures-util

Signed-off-by: Lucio Franco <[email protected]>

* fmt
2019-08-07 16:26:44 -04:00
Lucio Franco 6412389bba executor: update park implementation (#1402)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 13:01:13 -07:00
tmiasko 53a94c025d io: implement AsyncBufRead for &[u8] and Cursor (#1397)
* `impl AsyncRead for &[u8]`
* `impl AsyncBufRead for &[u8]`
* `impl<T: AsRef<[u8]> + Unpin> AsyncRead for Cursor<T>`
* `impl<T: AsRef<[u8]> + Unpin> AsyncBufRead for Cursor<T>`
2019-08-07 12:57:37 -07:00
Gurwinder Singh 7174c63bf9 codec: move length delimited codec to tokio-codec (#1401) 2019-08-07 12:10:05 -07:00
Ivan Petkov cb2336ff3d process: Misc polish (#1400)
* Denied all warnings in tests, and denied rust_2018_idioms violations
* Bumped the crate version and set publish = false
* Pruned dependencies:
 - Only pull in tokio-sync on windows where it is used
 - Removed unused dev-dependencies
* Switch to Async{Read, Write} traits from tokio-io rather than
futures-io
* Use #[tokio::test] where possible
* Removed deprecated items
* Fix all doc examples
2019-08-07 10:38:45 -07:00
Carl Lerche 47e2ff48d9 tokio: fix API doc examples (#1396) 2019-08-06 14:03:49 -07:00
Carl Lerche 2f43b0a023 sync: polish and update API doc examples (#1398)
- Remove `poll_*` fns from some of the sync types.
- Move `AtomicWaker` and `Lock` to the root of the `sync` crate.
2019-08-06 13:54:56 -07:00
Carl Lerche 05d00aebb7 uds: remove poll_* fns in favor of async fns (#1394) 2019-08-05 15:05:02 -07:00
Carl Lerche 62733a6594 udp: remove poll_* fns in favor of async fns (#1393)
This removes the need for manual futures.
2019-08-05 14:18:18 -07:00
Carl Lerche 6d8cc4e475 tcp: update API documentation (#1392) 2019-08-05 11:50:55 -07:00
Carl Lerche 6cbe3d4f82 fs: use async fn instead of custom futures (#1381)
Also update all the doc examples.
2019-08-04 11:24:30 -07:00
Carl Lerche 337646b97f tokio: re-export future/stream utils (#1387) 2019-08-03 21:08:29 -07:00
Taiki Endo 0bb015588a codec: add AsyncBufRead/BufRead implementations (#1385)
* AsyncBufRead for FramedWrite2<T>
* BufRead for FramedWrite2<T>
* AsyncBufRead for Fuse<T, U>
* BufRead for Fuse<T, U>
2019-08-03 20:15:50 -07:00
Steven Fackler 63377e2110 Add AsyncWriteExt::shutdown (#1382) 2019-08-03 00:51:24 -04:00
Carl Lerche 878503f965 docs: update API documentation for some crates (#1380)
Updates API documentation for

- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
2019-08-02 14:35:32 -07:00
Carl Lerche 2c01b3e0e0 io: remove util from default features (#1379)
Sub-crates should require opting into features.
2019-08-02 12:59:24 -07:00
Carl Lerche ee9105d166 tokio: add async io traits to prelude (#1378) 2019-08-02 12:50:40 -07:00
Lucio Franco 5a4f849bba tokio: update tinyhttp example to async/await (#1372) 2019-08-02 12:24:15 -07:00
Lucio Franco ff41108834 io: move io helpers back into tokio-io (#1377)
Utilities are made optional with a feature flag.
2019-08-02 12:23:44 -07:00
Lucio Franco 6b202722ea io: Add AsyncWriteExt::flush (#1376)
* io: Add `AsyncWriteExt::flush`

* fmt

* fix clippy
2019-08-02 13:53:49 -04:00
Lucio Franco 144d980e5c tokio: update connect to async/await (#1375) 2019-08-02 10:03:06 -07:00
Lucio Franco 81d789b88f tokio: Update proxy to async/await (#1373) 2019-08-01 20:03:34 -07:00
Lucio Franco 634c19582f chore: add rust-toolchain file to track nightly version (#1374) 2019-08-01 20:00:55 -07:00
Ivan Petkov ff922bbe6d signal: Change constructors to return a result instead of lazy future (#1340) 2019-07-30 18:23:26 -07:00
Gurwinder Singh bf38631d6a chore: Fix spelling mistake (#1359) 2019-07-30 10:53:11 -07:00
Taiki Endo 6dda866191 tokio: re-enable StreamExt (#1362) 2019-07-30 09:55:34 -07:00
Taiki Endo 03e450deb1 sync: switch branch of loom dev-dependency to master (#1367)
* sync: switch branch of loom dev-dependency to master

* replace loom::fuzz with loom::model
2019-07-30 10:11:46 -04:00
andy finch fbf90e6356 Update process to use std::future (#1343) 2019-07-29 18:36:11 -07:00
Shell Chen 74168ae82f tcp: add async fn TcpStream::peek (#1360)
* tcp: add `async fn TcpStream::peek`

* tcp: apply rustfmt on tests
2019-07-26 10:55:02 -04:00
John Doneth d038009e7d Update chat example to async/await (#1349) 2019-07-25 19:44:23 -04:00
John Doneth 132e9f1da5 Update examples to return Result (#1305)
* update echo-udp

* update echo

* update hello_world

* update udp-client

* rustfmt

* remove send & sync

* rebase & change new updated examples
2019-07-25 16:47:31 -04:00
Taiki Endo fe021e6c00 ci: enable clippy lints (#1335) 2019-07-26 03:47:14 +09:00
Lucio Franco f311ac3d4f buf: Inital pass at updating BufStream (#1355) 2019-07-25 14:21:48 -04:00
Shell Chen 298be80249 tokio: include async-trait feature for uds (#1352) 2019-07-25 08:07:33 -07:00
John Doneth 79b017c773 Export LinesCodecError (#1350) 2019-07-24 15:26:41 -04:00
Taiki Endo ca0e5cc670 add TryFrom/From implementations (#1347)
* TryFrom<net::TcpListener> for TcpListener
* TryFrom<net::TcpStream> for TcpStream
* TryFrom<net::UdpSocket> for UdpSocket
* TryFrom<net::UnixDatagram> for UnixDatagram
* TryFrom<net::UnixListener> for UnixListener
* TryFrom<net::UnixStream> for UnixStream
* TryFrom<UnixDatagram> for mio_uds::UnixDatagram
* TryFrom<File> for io::File
* From<io::File> for File
2019-07-24 09:26:12 -07:00
Douman 59bc364a0e macros: detect double test attribute (#1336) 2019-07-22 09:28:07 -07:00
Taiki Endo e88d10a3cb chore: bump to newer nightly (#1338) 2019-07-22 06:04:02 +09:00
Ivan Petkov a3b8d82711 Merge tokio-process into tokio
Original repo can be found at https://github.com/alexcrichton/tokio-process/
2019-07-21 11:08:16 -07:00
Ivan Petkov d9688bc094 signal: change unix::Signal to return () instead of signum (#1330)
* This simplifies the API surface by returning () instead of the signal
number that was used during registration. This also more closely mirrors
the cross-platform `CtrlC` event stream API
* This is a **breaking change**
2019-07-20 15:12:53 -07:00
Ivan Petkov 320a5fdca7 signal: replace windows::Event with windows::CtrlBreak (#1331)
* Add a new `windows::CtrlBreak` struct which wil represent a stream of
CTRL_BREAK_EVENT signals on Windows systems
* The `windows::Event` type is no longer publicly accessible and is
replaced by using `CtrlC` or `windows::CtrlBreak`.

[breaking-change]
2019-07-20 10:50:27 -07:00
Taiki Endo 9af07ce208 chore: remove redundant field names in struct literals (#1334) 2019-07-20 10:43:19 -07:00
Taiki Endo 1b2d997863 chore: use ptr::{null, null_mut} instead of 0 as *{const, mut} (#1333) 2019-07-20 10:41:02 -07:00
Taiki Endo 7a52ddcd09 chore: remove unnecessary conversion (#1332) 2019-07-20 12:10:42 -04:00
Carl Lerche 9d3e5aac08 tokio: remove Send + 'static requirement from block_on (#1329)
Removes the `Send` requirement to futures passed to `Runtime::block_on`.
Previously, `block_on` was implemented by sending the future to a
runtime thread. In order to do this, the future must be Send.

The reason why the future is sent to the pool is because we cannot
guarantee, while off the pool, that a reactor / timer thread is running.
This is due to a limitation in the current version of tokio-threadpool.
There is a plan to fix this (#1177), but the proper fix is non trivial.

In order to unblock APIs that require this, this patch updates the
runtime to spawn an always running thread containing a reactor and
timer. All calls to `block_on` will use that reactor and timer.
2019-07-19 17:25:04 -07:00
Carl Lerche a99fa6e096 chore: remove tokio-futures facade crate (#1327)
This switches from using the tokio-futures facade to referencing
futures-* crates directly.
2019-07-19 13:11:46 -07:00
David Kellum b89ed00a0d Remove last non-dev dependency on rand crate (#1324)
Use std RandomState for XorShift seeding. This allows dropping _rand_
crate dep here, accept as a dev dependency for tests or benchmarks.
2019-07-19 12:12:32 -07:00
Dylan Frankland 12ce75f088 fs: add remove_dir_all and RemoveDirAllFuture (#1325)
Adds the sister function to `remove_dir` and mirrors the `create_dir_all` that's already exposed.
2019-07-19 12:09:53 -07:00
Taiki Endo a88308ed9f tokio: add AsyncReadExt::read_to_string (#1326) 2019-07-19 11:50:00 -07:00
João Oliveira a298472da8 tokio-tls: enable Send and Sync (#1317)
-  update 0 as *mut () calls to std::ptr::null_mut()
- impl Send and Sync for AllowStd
2019-07-19 10:21:26 -07:00
Shell Chen d0bb16192b timer: change Into to From trait for Elapsed (#1322) 2019-07-17 09:04:59 -04:00
Shell Chen a18ddb3b61 timer: impl Into<std::io::Error> for Elpased (#1321)
That convert Elpased to ErrorKind::TimedOut
2019-07-16 20:22:03 -07:00
Jon Gjengset 003b4d8074 Get rid of Enter for with_default (#1315)
We want executors to enforce that there are never multiple active at the
same time. This is ensured through `Enter`, which will panic if you
attempt to create more than one. However, by requiring you to pass an
`&mut Enter` to `executor::with_default`, we were *also* disallowing
temporarily overriding the current executor.

This patch removes that requirement.
2019-07-16 14:29:35 -04:00
Yin Guanhao 6d186fe40e Replace (some) uninitialized with MaybeUninit (#1295) 2019-07-16 10:47:46 -07:00
Diggory Blake 0d99ddd4f4 tcp: implement "split_mut" for TcpStream (#1289) 2019-07-16 10:28:00 -07:00
João Oliveira 448d9d2eab tls: update to std-future (#1224) 2019-07-16 10:26:08 -07:00
David Kellum 0de3a69eb4 fs: drop deprecated tempdir crate use in tests (#1312)
In particular because it pulls in old rand duplicates. Replace use
with tempfile::tempdir() which has been available since tempfile
3.0.0.
2019-07-15 15:32:33 -07:00
John Doneth 61aee5fc28 examples: pdate tinydb example (#1288)
Update tinydb example to use async / await.
2019-07-15 15:19:36 -07:00
Sean McArthur 7f7f74985e io: Minor adjustments to tokio-test IO (#1306)
This also re-exports `bytes::{Buf, BufMut}` from `tokio-io`.
2019-07-15 14:53:16 -07:00
Jon Gjengset e6cf976662 tokio: include async-traits feature (#1314)
The `tokio` facade crate will depend on the `async-traits` feature flag in
sub crates.
2019-07-15 14:02:14 -07:00
Taiki Endo b14e189e44 add #[must_use] to more futures and streams (#1309) 2019-07-15 13:28:56 -07:00
Taiki Endo 2dde2b448f Fix import of ready macro 2019-07-15 11:52:13 -07:00
Taiki Endo 6742816e78 tokio: add AsyncBufReadExt::lines 2019-07-15 11:52:13 -07:00
Taiki Endo ab040bb498 tokio: add AsyncBufReadExt::read_line 2019-07-15 11:52:13 -07:00
Taiki Endo 0cfa120ba8 tokio: add AsyncBufReadExt::read_until 2019-07-15 11:52:13 -07:00
Taiki Endo 5774a9cd64 io: add AsyncBufRead trait 2019-07-15 11:52:13 -07:00
John Doneth da49ede41e update udp-codec example (#1293) 2019-07-15 14:14:03 -04:00
Carl Lerche d224d6415e chore: indicate the master branch docs are old. (#1304)
Fixes #1292
2019-07-15 10:44:47 -07:00
Gurwinder Singh 83273b8b50 chore: use ready macro from futures-core (#1300) 2019-07-15 10:43:54 -07:00
Taiki Endo ca708d6d87 chore: update rand dependency to 0.7 (#1302) 2019-07-15 10:13:10 -07:00
matthieugras 0b75c0c53d executor: block thread when needed in block fn (#1303)
Fix #1296
2019-07-15 08:56:20 -07:00
Alex Gaynor 5fbb36a060 reactor: bump parking_lot dependency (#1298) 2019-07-14 09:28:54 -07:00
Gurwinder Singh c897a5b696 Re-export tokio-fs (#1287) 2019-07-14 12:03:49 -04:00
Sean McArthur 48d7f7b931 tokio-test: add tokio_test::io mock builder 2019-07-12 11:19:12 -07:00
Carl Lerche 2291823181 tokio: re-export correct tokio-uds version (#1286)
An earlier PR (#1282) re-exported the version from crates.io and not git
master.
2019-07-11 10:27:51 -07:00
andy finch 795e02f4c6 fs: update to use std::future (#1269) 2019-07-11 09:05:49 -07:00
Carl Lerche 7ac8bfc821 chore: bump to newer nightly (#1284) 2019-07-10 14:36:36 -07:00
Carl Lerche a79483750f tokio: update echo example (#1283) 2019-07-10 14:21:20 -07:00
Carl Lerche 3855f373d3 tokio: re-export tokio-uds (#1282)
The tokio-uds crate has been previously updated to std::future. This
commit enables the re-export in the tokio facade crate.
2019-07-10 11:21:27 -07:00
Carl Lerche bd3f3270db tokio: update threaded runtime to std::future (#1280)
re-enables the threaded runtime and sets it (again) as the default.
2019-07-10 11:21:06 -07:00
Taiki Endo e5525628cd chore: remove usage of deprecated ONCE_INIT (#1281) 2019-07-10 08:35:07 -07:00
Carl Lerche f1b8a318d9 tokio: add AsyncReadExt::read_to_end (#1279) 2019-07-09 16:17:58 -07:00
Carl Lerche 64343f1b78 tokio: add AsyncWriteExt::write_all (#1277) 2019-07-09 12:37:14 -07:00
Ruben De Smet 82795184c1 tokio: rewrite examples with async. (#1228) 2019-07-09 11:21:12 -07:00
Thomas Lacroix f529928d87 chore: script updating versions in links to docs.rs (#1249) 2019-07-09 11:19:38 -07:00
Ivan Petkov 461eebe612 signal: Replace ctrl_c with a CtrlC struct (#1273)
* Add a new `CtrlC` struct which will represent a stream of SIGINT
signals on Unix or the CTRL_C event on Windows
* `CtrlC` implements `Stream<Output = ()>` rather than `IoSteam` as
previously
2019-07-09 08:48:46 -07:00
Gurwinder Singh 407d15cf93 chore: Add link to docs (#1276) 2019-07-09 11:21:29 -04:00
Yin Guanhao 80915906d8 uds: update to std-future (#1227) 2019-07-08 14:58:40 -07:00
Yin Guanhao 88e775dcf0 udp: UdpSocket split support (#1226) 2019-07-08 14:47:31 -07:00
Carl Lerche 8b49a1e05f chore: update examples link in README (#1274) 2019-07-08 13:34:39 -07:00
Carl Lerche 8fa1510d67 timer: fix build (#1275) 2019-07-08 11:23:15 -07:00
Reto Kaiser 7797a377c3 current-thread: make tokio_current_thread::Handle Sync (#1119) 2019-07-08 10:25:34 -07:00
Steven Fackler b62d224fac timer: fix Handle::timeout (#1093)
The old implementation didn't work for Timeout<Stream>, since the method
took a deadline rather than a timeout.
2019-07-08 10:18:37 -07:00
Aaron Hill d4803bc868 Use Sink trait from futures-sink-preview (#1244) 2019-07-08 09:56:11 -07:00
Thomas Lacroix e07a03b3c5 signal: update instructions in Ctrl-C example (#1270)
Fixes: #1248
2019-07-07 09:49:19 -07:00
Taiki Endo 7b86acb71d chore: Update futures-preview to 0.3.0-alpha.17 (#1267) 2019-07-04 14:34:57 -07:00
Steffen Butzer 0651f09427 Remove usage of deprecated std::error::Error methods (#1206) (#1245) 2019-07-03 23:06:03 -07:00
Thomas Lacroix 516251052d Add missing links in README.md (#1233)
Fixes: #1229
2019-07-03 22:59:10 -07:00
Ivan Petkov cbad83f362 signal: migrate to std::futures (#1218)
Migrate to std::futures and the futures 0.3 preview and use async/await
where possible

**Breaking change:** the IoFuture and IoStream definitions used to refer
to Box<dyn Future> and Box<dyn Stream>, but now they are defined as
Pin<...> versions which are technically breaking.

No other breaking or functional changes have been made
2019-07-03 10:40:59 -07:00
Eliza Weisman bd9760e124 add release documentation to CONTRIBUTING.md (#1171)
## Motivation

Currently, the process for releasing a new version of a Tokio crate is
somewhat complex, and is not well-documented. To make it easier for
contributors to release minor versions more frequently, there should be
documentation describing this process.

## Solution

This branch adds a section to `CONTRIBUTING.md` describing how to
release a new version of a Tokio crate. The steps are based on those
described by @carllerche in an offline conversation.

I've also added a quick shell script to actually publish new crate 
versions. This should make it harder to make mistakes when 
publishing.

Signed-off-by: Eliza Weisman <[email protected]>
2019-07-03 10:18:02 -07:00
Carl Lerche 3e898f58a5 tcp: add ascyc fn TcpListener::accept (#1242)
Refs: #1209
2019-07-03 09:49:56 -07:00
Ivan Petkov c531865d2c ci: don't generate docs for deps on FreeBSD (#1241) 2019-07-03 09:41:35 -07:00
Ivan Petkov 722eb257be ci: scope each tests/examples invocation to a specific crate (#1238) 2019-07-03 08:49:05 -07:00
Taiki Endo ceed29586b io: fix documents (#1231) 2019-07-01 20:44:12 -07:00
Carl Lerche 70eca184f0 tokio: re-enable timer in runtimes (#1237)
This also brings back the timer tests in the tokio crate.
2019-07-01 18:27:13 -07:00
Carl Lerche b2c777846e timer: finish updating timer (#1222)
* timer: restructure feature flags
* update timer tests
* Add `async-traits` to CI

This also disables a buggy `threadpool` test. This test should be fixed in the future.

Refs #1225
2019-06-30 08:48:53 -07:00
Lucio Franco 8e7d8af588 docs: add note in the readme about the master branch (#1230) 2019-06-29 21:47:20 -04:00
Yin Guanhao 7380dd2482 TcpSocket specialized split (#1217) 2019-06-28 23:36:49 -07:00
Eliza Weisman af46eac583 chore: remove tokio-trace, add "Related Projects" to README (#1221)
## Motivation

The `tokio-trace` and `tokio-trace-core` crates have been renamed to
`tracing` and `tracing-core`, and moved to their own repository
(`tokio-rs/tracing`).

## Solution

This branch removes `tokio-trace` and `tokio-trace-core` from the
`tokio` repository. In addition, I've added a "Related Projects" section
to the root README, which lists `tracing` (as well as  `mio`, and
`bytes`) as other libraries maintained by the Tokio project. I thought
that this would help folks looking for `tokio-trace` here find it in its
new home.

In addition, it changes `tokio` to depend on `tracing-core` rather than
`tokio-trace-core`.

Closes #1159

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-28 13:13:46 -07:00
Carl Lerche e7488d983e threadpool: update to std::future (#1219)
An initial pass at updating `tokio-threadpool` to `std::future`. The
codebase and tests both now run using `std::future` but the wake
mechanism is not ideal. Follow up work will be required to improve on
this.

Refs: #1200
2019-06-27 22:30:56 -07:00
Sean McArthur e4415d986a sync: change oneshot poll_close to poll_closed
The action of `Sender::poll_close` is to check if the receiver has been
closed, not to try to close the sender itself. So change to
`poll_closed`.
2019-06-27 13:56:58 -07:00
Carl Lerche ff906acdfb ci: disable cache on cirrus (#1215)
Caching takes longer than rebuilding
2019-06-27 12:08:43 -07:00
Carl Lerche 32ceccb465 sync: add async APIs to oneshot and mpsc (#1211)
Adds:

- oneshot::Sender::close
- mpsc::Receiver::recv
- mpsc::Sender::send

Also renames `poll_next` to `poll_recv`.

Refs: #1210
2019-06-27 11:33:36 -07:00
Douman 0af05e7408 macros: allow configuring runtime used by main macro (#1185) 2019-06-27 10:40:21 -07:00
jesskfullwood 6b9e7bdace codec: update to use std-future (#1214)
Strategy was to

- copy the old codec code that was temporarily being stashed in `tokio-io`
- modify all the type signatures to use Pin, as literal a translation as possible
- fix up the tests likewise

This is intended just to get things compiling and passing tests. Beyond that there is surely
lots of refactoring that can be done to make things more idiomatic. The docs are unchanged.

Closes #1189
2019-06-27 10:10:29 -07:00
Carl Lerche ed4d4a5353 chore: format code and enable rustfmt CI task (#1212) 2019-06-27 00:05:01 -07:00
Carl Lerche 1f47ed3dcc tokio: rewrite io_read.rs test to use async/await (#1207)
This simplifies the test
2019-06-26 17:06:56 -07:00
Carl Lerche e9aaacddbd tokio: re-export sync::{lock,mpsc} (#1208)
These types have been updated already.
2019-06-26 16:54:15 -07:00
Carl Lerche 11f6b2862f tokio: move I/O helpers to ext traits (#1204)
Refs: #1203
2019-06-26 14:42:19 -07:00
Carl Lerche 8404f796ac test: get cargo test --tests working (#1205)
Broken tests are disabled
2019-06-26 14:40:52 -07:00
Yin Guanhao 6316aa1d0b Update tokio-udp to use std-future (#1199) 2019-06-26 14:41:36 -04:00
Bhargav 0784dc2767 tokio: add read_exact method (#1202) 2019-06-26 11:36:09 -07:00
Denis dd126c2333 Implement TryFrom to transform various I/O primitives into their mio counterparts (#1158)
* `TryFrom<TcpListener> for mio::net::TcpListener`
* `TryFrom<TcpStream> for mio::net::TcpStream`
* `TryFrom<UdpSocket> for mio::net::UdpSocket`
* `TryFrom<UnixListener> for mio_uds::UnixListener`
* `TryFrom<UnixStream> for mio_uds::UnixStream`
2019-06-26 08:51:38 -07:00
Lucio Franco 3cc33dca7c sync: Fix lock test to actually test the inner lock value (#1197)
* sync: Fix lock test to actually test the returned value

* Update lock test to use task.poll
2019-06-26 11:32:41 -04:00
Carl Lerche dc5fa80a09 macros: re-export main macro from tokio (#1198)
Includes minor fixes and a very basic example.

Fixes #1183
2019-06-25 20:14:21 -07:00
Zahari Dichev 455782b964 trace: Allow setting event parents explicitly (#1109)
## Motivation 

As mentioned in tokio-rs/tracing#1100  it makes sense to be able to set
the parents of events explicitly.

## Solution 

For that to happen the Parent type is extracted from span.rs and a
`parent` field is added to Event. Additionally the appropriate macros
arms are added with corresponding tests as described in
tokio-rs/tracing#1100

Closes tokio-rs/tracing#1100

Signed-off-by: Zahari Dichev <[email protected]>
2019-06-25 15:12:52 -07:00
Lucio Franco 29e417c257 tokio: Add io copy, read, and write (#1187) 2019-06-25 16:51:49 -04:00
Ivan Petkov 9df1140340 signal: factor out event delivery into its own module to share between Unix and Windows (#1174)
Today the Unix and Windows implementations have similar yet differing
implementations of hooking into OS events and propagating them to any
listening futures. Rather than re-implement the same behavior two
different ways, we should factor out any commonality into a shared
module and keep the Unix/Windows modules focused solely on OS
integrations.

Reusing the same implementation across OS versions also allows for more
consistent behavior between platforms, which also makes squashing bugs
much easier.

This change introduces the `registry` module which handles creating and
initializing a global map of signals/events and their registered
listeners. Each OS specific module is expected to implement the OS hooks
which delegate to invoking the registry module's methods for
distributing the event notifications.

# Use registry module for Windows implementation

Note this still uses the same architecture as previously: a driver task
is spawned by the first registered event, and that task is responsible
for delivering any events to registered futures. (If that first event
loop goes away, all events will deadlock). A solution to this issue will
be explored at a later time.
2019-06-25 13:07:59 -07:00
Lucio Franco e2b4bdb647 sync: Add LockFuture for Lock (#1184) 2019-06-25 10:42:35 -07:00
Ivan Petkov c6defbce4b process: Move files to their own directory 2019-06-24 17:31:47 -07:00
Ivan Petkov b7846a4e2f process: Remove unneeded files 2019-06-24 17:31:00 -07:00
Ivan Petkov cb8607a816 process: Update to 2018 edition 2019-06-24 17:29:33 -07:00
Ivan Petkov 27c15471c1 process: Run cargo fmt 2019-06-24 17:29:33 -07:00
Ivan Petkov 0ab25878bd process: Update README 2019-06-24 17:29:32 -07:00
Eliza Weisman 448302c3d4 trace: Improve documentation (#1148) 2019-06-24 19:22:05 -05:00
Ivan Petkov 934a1467d4 process: Update CHANGELOG 2019-06-24 17:12:17 -07:00
Ivan Petkov 4d639e246b process: Update Cargo.toml 2019-06-24 17:10:58 -07:00
Ivan Petkov ff5381de8d process: Update license files 2019-06-24 17:10:58 -07:00
Ivan Petkov 061452dc01 process: Delete flaky and (now) unused test 2019-06-24 17:10:58 -07:00
Ivan Petkov a6b2682309 process: Bump to 0.2.4 2019-06-24 16:57:20 -07:00
Ivan Petkov cf84a59e5a process: Don't kill child on drop if already successfully killed 2019-06-24 16:57:20 -07:00
Ivan Petkov e90e33d5df process: Add unit tests for dropping killing dropped children 2019-06-24 16:57:20 -07:00
Ivan Petkov fa5da27d98 process: Utilize a global orphan process queue to avoid leaks 2019-06-24 16:57:20 -07:00
Ivan Petkov ecaa069f0f process: Implement a queue for repeatedly attempting to reap orphaned processes 2019-06-24 16:57:20 -07:00
Ivan Petkov fc15d7d4a4 process: Only pull in mio dependency on unix platforms 2019-06-24 16:57:20 -07:00
Ivan Petkov a70a3b599a process: ci: move cargo tool installation to after_success 2019-06-24 16:57:20 -07:00
Ivan Petkov 26faefcc34 process: ci: enable clippy checks as part of the build 2019-06-24 16:57:19 -07:00
Ivan Petkov f16725ea9f process: Fix clippy warnings 2019-06-24 16:57:19 -07:00
Ivan Petkov caf43221b5 process: ci: fix cargo binary caching 2019-06-24 16:57:19 -07:00
Ivan Petkov 93680357dd process: Fix drop_kills test when running on macOS with a single thread 2019-06-24 16:57:19 -07:00
Ivan Petkov 784d21ae31 process: Try pinning mio to 0.1.16 2019-06-24 16:57:19 -07:00
Ivan Petkov 0938ccfefd process: ci: cache cargo tarpaulin build 2019-06-24 16:57:19 -07:00
Ivan Petkov 6fa2fdab44 process: Ensure all tests are run with an explicit timeout 2019-06-24 16:57:19 -07:00
Ivan Petkov d0d13d0bd0 process: Change codecov comment behavior to default 2019-06-24 16:57:19 -07:00
Ivan Petkov 8a1777b800 process: Rename EventedReaper to Reaper 2019-06-24 16:57:18 -07:00
Ivan Petkov 42d0f53ddb process: Optimize out the "reaped" flag 2019-06-24 16:57:18 -07:00
Ivan Petkov db0c4147c8 process: Refactor Unix process handling 2019-06-24 16:57:18 -07:00
Ivan Petkov 10fd2afd18 process: Simplify child IO registration 2019-06-24 16:57:18 -07:00
Ivan Petkov 83a55601ef process: Move src/unix.rs to src/unix/mod.rs 2019-06-24 16:57:18 -07:00
Ivan Petkov b37120f61c process: Update line-by-line doc example to be more flexible 2019-06-24 16:57:18 -07:00
Ivan Petkov 91dbf24cf4 process: Update min supported rust version as per the Tokio project policy 2019-06-24 16:57:18 -07:00
Ivan Petkov c78fd6d6c5 process: Update Travis link from .org to .com 2019-06-24 16:57:18 -07:00
Ivan Petkov 025474dfbb process: ci: Install cargo-tarpaulin *after* initial tests 2019-06-24 16:57:18 -07:00
Ivan Petkov e7dfcf90fe process: ci: Enable code coverage tracking via codecov.io 2019-06-24 16:57:17 -07:00
Ivan Petkov ecdfe4c474 process: ci: collect code coverage info via cargo-tarpaulin 2019-06-24 16:57:17 -07:00
Ivan Petkov 37b4efb9e2 process: Bump version to 0.2.3 2019-06-24 16:57:17 -07:00
Ivan Petkov c94f607f1b process: Fix some test case deprecation warnings 2019-06-24 16:57:17 -07:00
Ivan Petkov 76438c9e70 process: Implement AsRawHandle for ChildStd{in, out, err} for parity 2019-06-24 16:57:17 -07:00
Yuya Nishihara e0e9594f71 process: Implement AsRawFd for ChildStd* structs 2019-06-24 16:57:17 -07:00
Yuya Nishihara 3b43262a10 process: Implement AsRawFd for inner Fd<T> wrappers and use it instead of self.0 2019-06-24 16:57:17 -07:00
Ivan Petkov 5f18bf669f process: Bump minimum supported rustc version to 1.26 2019-06-24 16:57:17 -07:00
Ivan Petkov 1581c8b475 process: Bump minimum required version of tokio-signal to 0.2.5 2019-06-24 16:57:16 -07:00
Ivan Petkov d3b2efc815 process: Add regression test for signal starvation 2019-06-24 16:56:53 -07:00
Ivan Petkov f7c4e3cd84 process: Bump min supported rustc version to 1.25 2019-06-24 16:56:53 -07:00
Ivan Petkov 329ad3324c process: Bump to 0.2.2 2019-06-24 16:56:53 -07:00
Ivan Petkov 2b6695d25a process: Update CHANGELOG 2019-06-24 16:56:53 -07:00
Ivan Petkov 9290602815 process: Unix: preregister for signal notifications before polling child 2019-06-24 16:56:53 -07:00
Ivan Petkov 827e77e71e process: Bump to 0.2.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 7b3e4b98ac process: Update Child::forget example to use the tokio runtime 2019-06-24 16:56:52 -07:00
Ivan Petkov 5e9d60e834 process: Add a CHANGELOG 2019-06-24 16:56:52 -07:00
Ivan Petkov 8270965459 process: Remove dependency on tokio-core 2019-06-24 16:56:52 -07:00
Ivan Petkov e6b044a820 process: Bump tokio-signal version to 0.2 2019-06-24 16:56:52 -07:00
Ivan Petkov de9b401457 process: Mark status_async2/StatusAsync2 as deprecated 2019-06-24 16:56:52 -07:00
Ivan Petkov ad5179b2d5 process: Remove all items deprecated in 0.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 0aceba21bd process: Bump to 0.1.6 2019-06-24 16:56:52 -07:00
Ivan Petkov 09e21eceea process: Unix: mark child as reaped on kill 2019-06-24 16:56:52 -07:00
Arvid E. Picciani bdc87856f2 process: fix zombification on Drop on unix 2019-06-24 16:56:51 -07:00
Ivan Petkov 7987b64445 process: Clarify that Child::forget docs that it can leak OS resources 2019-06-24 16:56:51 -07:00
Alex Crichton 32c928b607 process: Bump to 0.1.5 2019-06-24 16:56:51 -07:00
Alex Crichton 82aeae147d process: Update dev-dependencies 2019-06-24 16:56:51 -07:00
Alex Crichton f48944c1fb process: Update winapi to 0.3 2019-06-24 16:56:51 -07:00
Ivan Petkov f0680617ee process: Fix project name typo in README 2019-06-24 16:56:51 -07:00
Alex Crichton dbc185cd3a process: Tweak travis config 2019-06-24 16:56:51 -07:00
Alex Crichton c205e2c358 process: Fix copy/paste 2019-06-24 16:56:51 -07:00
Alex Crichton acec6356ee process: Clarify wording of license information in README. 2019-06-24 16:56:51 -07:00
Alex Crichton c11eec3908 process: Bump to 0.1.4 2019-06-24 16:56:50 -07:00
Alex Crichton 69295fac1e process: Add an Errors section to status_async2 2019-06-24 16:56:50 -07:00
Ivan Petkov b9c6eb309c process: Add status_async2 as a closer analog to spawn_async 2019-06-24 16:56:50 -07:00
Ivan Petkov 56d3914675 process: Bugfix: ensure status_async closes child's stdio handles after spawning 2019-06-24 16:56:50 -07:00
Ivan Petkov 34e71fa71a process: Add must_use annotations to all futures 2019-06-24 16:56:50 -07:00
Ivan Petkov 914b803429 process: Add Debug impls for nondeprecated structs 2019-06-24 16:56:50 -07:00
Alex Crichton 4d11784b01 process: Tweak docs and macro imports 2019-06-24 16:56:50 -07:00
Michael Pankov 50cabae181 process: Add an example with reading input line-by-line 2019-06-24 16:56:50 -07:00
Alex Crichton c101e9e11d process: Use appveyor to download rustup 2019-06-24 16:56:49 -07:00
Alex Crichton 5c5f793ef0 process: Bump to 0.1.3 2019-06-24 16:56:49 -07:00
Alex Crichton 1384b31d60 process: Update to tokio-io, mio, and tokio-core changes 2019-06-24 16:56:49 -07:00
Alex Crichton 521dc94021 process: Bump to 0.1.2 2019-06-24 16:56:49 -07:00
Alex Crichton ed23a06fb1 process: Update doc urls and metadata 2019-06-24 16:56:49 -07:00
Alex Crichton 1aee22505a process: Remove caveat about tokio-signal 2019-06-24 16:56:49 -07:00
Alex Crichton 01b5bf6761 process: Use join3 instead of two joins 2019-06-24 16:56:49 -07:00
Alex Crichton 6638cbc80e process: Update README 2019-06-24 16:56:49 -07:00
Alex Crichton 22bc5e2738 process: Bump back to 0.1.1 2019-06-24 16:56:49 -07:00
Alex Crichton a0c162c0ff process: Hide compat from docs 2019-06-24 16:56:48 -07:00
Alex Crichton ca51ae9651 process: Add back in 0.1.0 compatibility layer 2019-06-24 16:56:48 -07:00
Alex Crichton f3f99b723f process: Bump to 0.2.0 2019-06-24 16:56:48 -07:00
Alex Crichton f20e7a4d2b process: Bump minimum version of tokio-core 2019-06-24 16:56:48 -07:00
Alex Crichton 4a92c4d4b6 process: Tweak drop_kills test 2019-06-24 16:56:48 -07:00
Alex Crichton 9680ecc109 process: Share init in tests 2019-06-24 16:56:48 -07:00
Alex Crichton 124391e42b process: Add a simple wait_with_output test 2019-06-24 16:56:48 -07:00
Alex Crichton 6150be189f process: Rewrite the crate with an extension trait 2019-06-24 16:56:48 -07:00
Ivan Petkov ca9586a089 process: Add documentation to public declarations 2019-06-24 16:56:47 -07:00
Ivan Petkov 89b9792931 process: Update README with crates.io info 2019-06-24 16:56:47 -07:00
Alex Crichton a0cc60153a process: Fix nightly tests 2019-06-24 16:56:47 -07:00
Alex Crichton 7f3f868b66 process: Add Windows support for stdio streams 2019-06-24 16:56:47 -07:00
Andreas Rottmann 97ebb2275c process: [WIP] Actually be non-blocking 2019-06-24 16:56:47 -07:00
Andreas Rottmann 849a5ad0b2 process: Add support for stdio streams 2019-06-24 16:56:47 -07:00
Alex Crichton b16a8613b1 process: Test on stable 2019-06-24 16:56:47 -07:00
Alex Crichton 5664660156 process: Fix tests on nightly 2019-06-24 16:56:47 -07:00
Alex Crichton 4416ea07d8 process: Update travis token 2019-06-24 16:56:47 -07:00
Alex Crichton 56222c588b process: pass --target on appveyor 2019-06-24 16:56:46 -07:00
Alex Crichton 72179d49c5 process: Update to crates.io versions of deps 2019-06-24 16:56:46 -07:00
Alex Crichton 073a1a251a process: Track tokio-core master 2019-06-24 16:56:46 -07:00
Alex Crichton 31c81faf96 process: Add appveyor to readme 2019-06-24 16:56:46 -07:00
Alex Crichton 5e68b0d51d process: Don't build on stable, start w/ beta for now 2019-06-24 16:56:46 -07:00
Alex Crichton 4bd07ac6aa process: Add metadata info 2019-06-24 16:56:46 -07:00
Alex Crichton f4f7bb232e process: Fix a test on Windows 2019-06-24 16:56:46 -07:00
Alex Crichton 413e1b78a7 process: Fix a segfault on windows 2019-06-24 16:56:46 -07:00
Alex Crichton 649fa13a15 process: Remove unused imports 2019-06-24 16:56:45 -07:00
Alex Crichton eef655f3b1 process: Add a Windows implementation 2019-06-24 16:56:45 -07:00
Alex Crichton 97508096fa process: Initial commit 2019-06-24 16:56:41 -07:00
Carl Lerche 06c473e628 Update Tokio to use std::future. (#1120)
A first pass at updating Tokio to use `std::future`.

Implementations of `Future` from the futures crate are updated to implement
`Future` from std. Implementations of `Stream` are moved to a feature flag.

This commits disables a number of crates that have not yet been updated.
2019-06-24 12:34:30 -07:00
James Gilles aa99950b9c trace: Switch benchmarks to criterion (#1163)
Extracted from #1152

This makes it possible to run benchmarks on stable + gives more statistical reliability.
2019-06-24 12:05:45 -07:00
Takanori Ishibashi aac6998c22 chore: fix url in docs (#1173) 2019-06-24 07:33:46 -04:00
Matt Bilker df2c3cd475 trace: fix debug and debug_span macro regression from #1103 (#1170)
PR #1103 accidentally changed the log level for the debug and
debug_span macros to use the INFO level instead of the DEBUG
level. This PR corrects this regression back to the intended
behavior.
2019-06-22 16:31:25 -07:00
James Gilles 36ed35c52c trace: add program-wide default dispatcher (#1152)
## Motivation

I was just trying to use tokio-trace for a greenfield project, but I was frustrated to discover that I couldn't really use it easily.

I was using the [`runtime`](https://docs.rs/runtime/0.3.0-alpha.4/runtime/) crate, which transparently spawns a thread pool executor for futures. In that thread pool, there's no way to set a tokio-trace subscriber for the duration of each thread, since you don't control the thread initialization. You *might* be able to wrap every future you spawn with a subscriber call, but that's a lot of work.

I was also confused because the documentation said that setting a subscriber in the main thread would use that subscriber for the rest of the program. That isn't the case, though -- the subscriber will be used only on the main thread, and not on worker threads, etc.

## Solution

I added a function `set_global_default`, which works similarly to the `log` crate:

```rust
tokio_trace::subscriber::set_global_default(FooSubscriber::new());
```

The global subscriber (actually a global `Dispatch`) is a `static mut` protected by an atomic; implementation is copied from the `log` crate. It is used as a fallback if a thread has no `Dispatch` currently set. This is extremely simple to use, and doesn't break any existing functionality.

Performance-wise, thread-local `Dispatch` lookup goes from ~4.5ns to ~5ns, according to the benchmarks. So, barely any runtime overhead. (Presumably there's a little compile-time overhead but idk how to measure that.) Since the atomic guard is only ever written once, it will be shared among a CPU's cores and read very cheaply.

I added some docs to partially address #1151. I also switched the tokio-trace benchmarks to criterion because the nightly benchmarks weren't compiling (missing `dyn` flags?)
2019-06-21 16:49:53 -07:00
Eliza Weisman 5925ca7720 trace: fix level_span macros not propagating parents (#1167)
Currently, when the `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!`, and `error_span!` macros are invoked with an explicit
parent, a name, and zero or more fields (no target), the macros don't
pass along the explicitly provided parent when expanding to the `span!`
macro. This is likely due to an oversight on my part.

This branch fixes these macros by adding the parent into the `span!`
macro expansion. I've also added a test to catch regressions

Shoutout to @jonhoo for catching this one!

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-21 11:17:06 -07:00
Max Bruckner 2ac132fb46 runtime: better error message in block_on_all on panics (#1166) 2019-06-21 11:10:58 -04:00
Hung-I Wang f9a0cb8792 timer: Implement Default for DelayQueue (#1118) 2019-06-21 10:42:52 -04:00
Igor Gnatenko 9fa6092e5a chore: Update parking_lot to 0.8 (#1078) 2019-06-21 10:42:09 -04:00
Eliza Weisman d4adeeef2f trace: Remove the AsId trait (#1145)
While we're making breaking changes to `tokio-trace`, it would be good
to get rid of the `AsId` trait. The goal of span functions that are
generic over `Span`/`Id` can be achieved without the unnecessary
complexity of defining a new trait. This would also make the API added
to `tokio_trace_core::Event` in #1109 more consistent with the
`tokio-trace::Span` API.

This branch removes `AsId` from `tokio-trace` and replaces its uses with
`impl Into<Option<Id>>` and `impl Into<Option<&'a Id>>`. While `AsRef`
might be more semantically correct for the borrowed-`Id` conversion, its
signature doesn't permit conversion into an `Option`. Implementations of
`Into<Option<Id>>` and `Into<Option<&'a Id>>` have been added for 
`tokio_trace::Span`.

This is _technically_ a breaking API change, as it changes function
signatures. However, the existing macro syntax still works as-is, and
the tests which pass `&Id`, `&Span`, and `&Option<Id>` to the span
macros all still compile after this change.

Closes #1143

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-13 12:53:08 -07:00
Steven Fackler 4f6395b31c Make threadpool::Runtime methods take &self (#1140)
The runtime is inherently multi-threaded, so it's going to have to deal
with synchronization when submitting new tasks anyway. This allows a
runtime to be shared by multiple threads more easily when e.g. building
a blocking facade over a tokio-based API.
2019-06-10 12:54:27 -07:00
yanjhk 5c0b56278b Use ThreadPool's impl of spawn (#1139) 2019-06-10 11:23:12 -07:00
Eliza Weisman 41ca9a43de trace: Add shorthand syntax for local fields (#1103)
## Motivation

A common pattern in `tokio-trace` is to use the value of a local
variable as a field on a span or event. Currently, this requires code
like:
```rust
info!(foo = foo);
```
which is not particularly ergonomic given how commonly this occurs.
Struct initializers support a shorthand syntax for fields where the name
of the field is the same as a local variable, and `tokio-trace` should
as well.

## Solution

This branch adds support for syntax like
```rust
let foo = ...;
info!(foo);
```
and 
```rust
let foo = Foo {
    bar: ...,
    ...
};
info!(foo.bar)
```
to the `tokio-trace` span and event macros. This syntax also works with
the `Debug` and `Display` field shorthand.

The span macros previously used a field name with no value to indicate 
an uninitialized field. A new issue, #1138, has been opened for finding a
replacement syntax for uninitialized fields. Until then, the `tokio-trace` 
macros will no longer provide a way to create fields without values, 
although the `-core` API will continue to support this.

Closes #1062 

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-09 13:16:35 -07:00
Carl Lerche 8d0f102de8 Merge branch 'v0.1.x' into merge-0.1 2019-06-05 12:28:39 -07:00
Kevin Leimkuhler 5dcb379f6d Bump tokio-sync to 0.1.6 (#1123) 2019-06-05 12:19:06 -07:00
Kevin Leimkuhler 970f75f830 sync: Add Sync impl for Lock (#1117) 2019-06-04 17:04:35 -07:00
Kevin Leimkuhler 619efed28b sync: Add Sync impl for Lock (#1116)
Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-06-03 11:12:28 -07:00
Carl Lerche 18ed0be851 executor: remove unnecessary APIs from Enter. (#1115) 2019-05-31 11:11:10 -07:00
Carl Lerche 01052f930a Bump tokio version to v0.1.21. (#1113) 2019-05-30 14:39:30 -07:00
Lucio Franco 940f2c3431 Update tokio-trace-core to 0.2 (#1111)
Also includes 1b498e8aa2
2019-05-30 11:33:55 -07:00
Eliza Weisman 84d5a7f5a0 trace: Change Span::enter to return a guard, add Span::in_scope (#1076)
## Motivation

Currently, the primary way to use a span is to use `.enter` and pass a
closure to be executed under the span. While that is convenient in many
settings, it also comes with two decently inconvenient drawbacks:

 - It breaks control flow statements like `return`, `?`, `break`, and
   `continue`
 - It require re-indenting a potentially large chunk of code if you wish
   it to appear under a span

## Solution

This branch changes the `Span::enter` function to return a scope guard 
that exits the span when dropped, as in:
```rust
let guard = span.enter();

// code here is within the span

drop(guard);

// code here is no longer within the span
```
The method previously called `enter`, which takes a closure and 
executes it in the span's context, is now called `Span::in_scope`, and
was reimplemented on top of the new `enter` method. 

This is a breaking change to `tokio-trace` that will be part of the
upcoming 0.2 release.

Closes #1075 

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-24 15:24:13 -07:00
Carl Lerche 1b498e8aa2 Fix TCP poll_hup test (#1106)
This updates tests to track a fix applied in Mio. Previously, Mio
incorrectly fired HUP events. This was due to Mio mapping `RDHUP` to
HUP. The test is updated to correctly generate a HUP event.

Additionally, HUP events will be removed from all platforms except for
Linux. This is caused by the inability to reliably map kqueue events to
the epoll HUP behavior.
2019-05-24 14:08:07 -07:00
Eliza Weisman b2c53987d9 trace: Add shorthand for field::display and field::debug (#1088)
## Motivation

In `tokio-trace`, field values may be recorded as either a subset of
Rust primitive types or as `fmt::Display` and `fmt::Debug`
implementations. Currently, `tokio-trace` provides the `field::display`
and `field::debug` functions which wrap a type with a type that
implements `Value` using the wrapped type's `fmt::Display` or
`fmt::Debug` implementation. However, importing and using these
functions adds unnecessary boilerplate. 

In #1081, @jonhoo suggested adding shorthand syntax to the macros,
similar to that used by the `slog` crate, as a solution for the
wordiness of the current API.

## Solution

This branch adds `?` and `%` sigils to field values in the span and
event macros, which expand to the `field::debug` and `field::display`
wrappers, respectively. The shorthand sigils may be used in any position
where the macros take a field value.

For example:
```rust
trace_span!("foo", my_field = ?something, ...); // shorthand for `debug`
info!(foo = %value, bar = false, ...) // shorthand for `display`
```

Adding this shorthand required a fairly large change to how field
key-value pairs are handled by the macros --- since `%foo` and `%foo`
are not valid Rust expressions, we can no longer match repeated 
`$ident = $expr` patterns, and must now match field lists as repeated
token trees. The inner helper macros for constructing `FieldSet`s and
`ValueSet`s have to parse the token trees recursively. This added a
decent chunk of complexity, but fortunately we have a large number of
compile tests for the macros and I'm quite confident that all existing
invocations will still work.

Closes #1081

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-21 10:31:48 -07:00
Carl Lerche 38092010c4 Merge branch 'v0.1.x' 2019-05-14 11:50:44 -07:00
Carl Lerche 475dabe96d Release tokio v0.1.20, tokio-timer v0.2.21, and remove async-await-preview feature. (#1089)
The `async-await-preview` feature is removed as 0.1 will no longer track
Rust nightly.

This also bumps:
- tokio-timer (0.2.11).
2019-05-14 11:21:24 -07:00
Carl Lerche cb4aea394e Update Tokio to Rust 2018 (#1082) 2019-05-14 10:27:36 -07:00
Jeehoon Kang 79d8820050 Fix link in tokio-futures/README.md (#1085)
`tokio-futures/README.md`'s link to the examples was wrong.
2019-05-10 10:19:32 -07:00
Carl Lerche 951f2fd910 test: re-export macro dependencies (#1077)
Callers may not always have `futures` available at the root of the
crate. Re-exporting dependencies makes them available to the macro at a
deterministic location.
2019-05-03 20:43:40 -07:00
Carl Lerche 4ef736b9d5 async-await: add current_thread::Runtime::block_on_async (#1072)
This function is used by the Tokio macros introduced by #1058  but was
omitted from the PR.
2019-04-30 19:55:22 -07:00
Steven Fackler 219f24cbf1 timer: Replace Handle::deadline with Handle::timeout (#1074)
Deadline was deprecated a while ago and replaced with Timeout, but the
methods on Handle got missed.

Fixes #1071
2019-04-30 10:29:54 -07:00
Carl Lerche ea282efb2e ci: fix isRelease condition (#1066) 2019-04-29 10:37:00 -07:00
Michal 'vorner' Vaner 042224d33c signal: Smaller dependency (#1069)
The signal-hook library got split into lower-level and higher-level
parts. The tokio-signal uses only API from the lower-level one, so it
can depend on it directly.

The only effect of this change is smaller amount of compiled (and
unused) code during compilation. There's no change in the code actually
used.
2019-04-28 19:12:40 -07:00
Ian Hamlin 927eb80ad4 Fix an error in the mit-url in the README.md (#1068) 2019-04-27 12:56:40 -07:00
Carl Lerche 6a8934e897 Fix threadpool dependency (#1061) 2019-04-25 22:23:24 -04:00
Carl Lerche 0e400af78c Async/await polish (#1058)
A general refresh of Tokio's experimental async / await support.
2019-04-25 22:22:32 -04:00
Carl Lerche df702130d6 tcp: fix some tests that spuriously fail (#1060)
This does not remove all cases of using a fixed port in doc tests, but
removing some should reduce the likelihood of spurious failures.
2019-04-25 12:01:39 -07:00
Carl Lerche 949adbb887 chore: remember to remove path deps on release (#1057) 2019-04-24 10:42:39 -07:00
Ryan Dahl b2b796a228 rt: forward panic_handler to tokio::runtime::Builder (#1055) 2019-04-24 10:41:42 -07:00
Igor Gnatenko abb014efc2 tokio: Bump min version of tokio-sync (#1054)
It is needed for lock functionality which tokio now uses.
2019-04-24 08:24:59 -07:00
Lucio Franco e5cf0cc717 Introduce tokio-test crate (#1030) 2019-04-23 20:17:57 -07:00
Carl Lerche 62f34e15ce Bump tokio to 0.1.19. (#1053)
This also bumps:

- tokio-async-await (0.1.7)
- tokio-buf (0.1.1)
- tokio-sync (0.1.5)
- tokio-threadpool (0.1.14)
2019-04-22 15:12:25 -07:00
Eliza Weisman 3ebca76a9a trace: prepare tokio-trace for release (#1051) 2019-04-22 14:15:07 -07:00
Ryan Dahl fea1f780bc threadpool: add panic_handler (#1052) 2019-04-21 16:26:09 -07:00
Eliza Weisman 712ca84033 trace-core: prepare for 0.2 release (#1047) 2019-04-21 10:20:28 -07:00
Jon Gjengset cf06621998 tokio-sync: Add async mutual exclusion primitive (#964)
This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.

This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
2019-04-18 13:16:26 -04:00
Lucio Franco 7e51ab05e9 buf: Add IntoStream (#1048)
* buf: Add IntoStream

* Add debug implementation for IntoStream

* Add get_ref, get_mut and into_inner
2019-04-18 11:42:34 -04:00
Eliza Weisman 4bfa4ffcdf trace-core: Dispatchers unset themselves (#1033)
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans. 

Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.

Before:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          51 ns/iter (+/- 12)
test span_repeatedly         ... bench:       5,073 ns/iter (+/- 1,528)
test span_with_fields        ... bench:          56 ns/iter (+/- 49)
test span_with_fields_record ... bench:         363 ns/iter (+/- 61)
```

After:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          35 ns/iter (+/- 12)
test span_repeatedly         ... bench:       4,165 ns/iter (+/- 298)
test span_with_fields        ... bench:          48 ns/iter (+/- 12)
test span_with_fields_record ... bench:         363 ns/iter (+/- 91)
```

Closes #1032 

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-16 15:51:45 -07:00
Taiki Endo 88b942652c async-await: fix examples (#1050)
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
  on Rust 2018
* Fix an example in the documentation
2019-04-16 14:19:38 -04:00
Taiki Endo 5029e80a89 async-await: update to new futures_api (#1049) 2019-04-16 14:07:03 -04:00
Eliza Weismanandcsmoe 847fb59b17 trace-core: Introduce callsite classification in metadata (#1046)
## Motivation

To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.

## Solution

A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.

`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.

New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.

Fixes: #986
Closes: #1008

Co-Authored-By: csmoe <[email protected]>
2019-04-11 11:56:42 -07:00
Jane Lusby b4fe517a16 trace-core: add a function to rebuild cached interest (#1039)
## Motivation

Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.

However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`.  Even when filters change very
infrequently, they must still always be re-evaluated every time.

In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.

## Solution

This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.

## Breaking Change

In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.

Closes #1038

Co-Authored-By: yaahallo <[email protected]>
2019-04-10 13:51:05 -07:00
Simon Wollwage 7ae010f0f3 async-await: Use Context instead of Waker in poll (#1041)
Rust nightly std::future::Future recently changed Waker
to Context.

Change to use Context

Co-Authored-By: Kintaro <[email protected]>
2019-04-10 09:14:31 -07:00
Carl Lerche 9144b2ff53 sync: remove unnecessary imports (#1043) 2019-04-09 12:26:11 -07:00
Lev Eniseev 2c4549a18a Add example of blocking environment (#1036) 2019-04-09 12:10:15 -07:00
João Oliveira 4f819b7cd1 trace: fix counters example, Span IDs must be greater than zero (#1037)
## Motivation
tokio-trace counter example was panicking due to returning 0 as the first Span Id

## Solution
start ID's with 1 instead
2019-04-05 13:51:20 -07:00
Eliza Weisman 197f88f3bc trace: Change Span::enter and record to take &self (#1029)
## Motivation

The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.

Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
    span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
    span.record("bar" &false);
});
```

Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.

The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.

## Solution

This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows. 

In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`. 

## Notes

In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.

Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.

Before:
```
test enter_span              ... bench:          13 ns/iter (+/- 1)
```

...and after:
```
test enter_span              ... bench:           3 ns/iter (+/- 1)
```

Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.

I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 15:06:47 -07:00
Eliza Weisman 44f65afcc6 trace: Allow field names to be separated by .s (#1027)
## Motivation

In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.

## Solution

This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.

## Notes

The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.

Closes #1018.
Closes #1022.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 14:44:11 -07:00
Eliza Weisman 4271a9cd8d trace: Fix subscriber benchmarks panicking (#1028)
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-02 13:42:31 -07:00
Matthias Prechtl 9d8096b911 Improve documentation of Subscriber::record and Subscriber::event (#1026) 2019-04-02 12:44:01 -07:00
João Oliveira 597f271c08 trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation 

Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952). 

Closes #1013

## Solution 

Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels

## Notes 

I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
2019-04-02 11:29:23 -07:00
Taiki Endo 599955f716 Replace try! macro with ? operator (#1024) 2019-04-01 13:45:59 -07:00
Ivan Petkov 91bb0f73f5 signal: refactor Windows registrations to be lazy (#1001)
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
  so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
  when calling `global_init`

Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS

Fixes #999
2019-04-01 12:46:22 -07:00
Eliza Weisman 6c9d8abba9 trace: Make Span API functions taking IDs a little more flexible (#1021)
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.

Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-01 11:42:29 -07:00
Carl Lerche 824b7b6759 buf: stream and iter helpers (#1011) 2019-03-29 12:26:13 -07:00
Carl Lerche cb91dd274a buf: impl Error for CollectVecError (#1010) 2019-03-29 08:49:08 -07:00
Eliza Weisman ea7178b8c6 trace-core: Add slightly more useful debug impls (#1014)
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.

When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.

The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-28 14:47:06 -07:00
Carl Lerche a99b8e2e0b buf: impl size_hint for str types + reorg tests (#1012) 2019-03-28 14:02:45 -07:00
Carl Lerche 03859a7dcd buf: implement FromBufStream for Bytes (#1009) 2019-03-27 19:22:06 -07:00
Eliza Weisman d8177f81ac trace: Allow trace instrumentation to emit log records (#992)
## Motivation

`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.

Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.

## Solution

This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation. 

## Notes

In order to allow spans to log their names when they are entered and 
exited even when the span is disabled, this branch adds an 
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.

Fixes: #949

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-26 16:43:05 -07:00
Son ceca2a3cd6 chore: add license to tokio (#1006) 2019-03-26 08:41:41 -07:00
Son 1524ee4b60 trace: Add static level filtering (#987)
## Motivation

`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.

## Solution

We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.

Closes #959
2019-03-25 15:19:20 -07:00
MOZGIII 7793d63739 Corrected doc for tokio_buf SizeHint (#1003) 2019-03-25 16:54:49 -04:00
Red Hara e0e26bc223 Fix typo in README.md in examples (#1002) 2019-03-24 13:02:11 -04:00
Eliza Weisman 9c5cad037f trace-core: Add overrideable downcasting to Subscribers (#974)
## Motivation

In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to 
downcast the current `Dispatch` to a concrete subscriber type.

For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust

pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
    tokio_trace::dispatcher::with(|c| {
        if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
            s.do_elizatracing_correlation_magic(tt, et);
        }
    }); 
}
```

This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).

## Solution

This branch makes the following changes:
 * Add an object-safe `downcast_raw` method to the `Subscriber` trait,
   taking a `TypeId` and returning an `*const ()` if the type ID 
   matches the subscriber's type ID, or `None` if it does not, and
 * Add `is<T>` and `downcast_ref<T>` functions to `Subscriber` 
   and `Dispatch`, using `downcast_raw`.

Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_ 
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour 
somehow, can downcast to the inner type if they choose to.

[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204

Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 16:21:46 -07:00
Eliza Weisman 30330da11a chore: Fix examples not working with cargo run (#998)
* chore: Fix examples not working with `cargo run`

## Motivation

PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:

```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
error: no example target named `chat`

Did you mean `echo`?
```

## Solution

This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:

```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
   Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
   Compiling tokio-reactor v0.1.9
   Compiling tokio-threadpool v0.1.13
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.10
   Compiling tokio-uds v0.2.5
   Compiling tokio-udp v0.1.3
   Compiling tokio-tcp v0.1.3
   Compiling tokio-fs v0.1.6
   Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
    Finished dev [unoptimized + debuginfo] target(s) in 7.04s
     Running `target/debug/examples/chat`
server running on localhost:6142
```

Signed-off-by: Eliza Weisman <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 15:25:42 -07:00
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
Carl Lerche 11e2af66a8 Bump Tokio to v0.1.15. (#869)
Also bumps:

- tokio-sync (0.1.0)
- tokio-threadpool (0.1.11)
- tokio-timer (0.2.9)
2019-01-25 10:20:09 -08:00
Carl Lerche a4aae1459c chore: move enumerate test to correct location (#867) 2019-01-24 20:47:45 -08:00
Zahari Dichev 12546d1d9c tokio-timer: fix DelayQueue bug when inserting shorter delay (#863)
Reset the delay of the queue in case an item that expires sooner than the last inserted is put
into the queue.
2019-01-24 14:36:41 -08:00
Zahari Dichev fbad6297c5 Add enumerate combinator to Stream (#832) 2019-01-24 11:50:34 -08:00
Jon Gjengset 0ec8986b0b Make reason for try_send errors clearer (#864) 2019-01-23 15:06:24 -08:00
Jon Gjengset c6f8bdb249 Remove T: Debug bound on mpsc Debug impls (#866)
Following from https://github.com/tokio-rs/tokio/pull/865, this PR
removes `#[derive(Debug)]` on `mpsc` sender and receiver types in favor
of explicit `impl fmt::Debug` blocks that don't have a `T: fmt::Debug`
bound.
2019-01-23 18:04:00 -05:00
Jon Gjengset c6f9a069a5 Explicit impl Clone for tx to avoid T: Clone (#865)
`#[derive(Clone)]` on a type `struct Foo<T>` adds an impl that requires that
`T: Clone`:

```rust
impl<T: Clone> Clone for Foo<T>
```

which is unfortunate in the case of senders, because we don't want to require
that the items being sent are `Clone` for the channel sender to be `Clone`.
This PR adds an explicit `impl Clone` for the bounded and unbounded sender
types which does not have the `T: Clone` bound.

Note that this is _also_ an issue with `#[derive(Debug)]`, but that one is
harder to work around as `chan::Tx` _also_ has `#[derive(Debug)]`, as does
`chan::Chan`, so we'd have to add explicit impls for all of them to make
progress.
2019-01-23 15:51:44 -05:00
Sean McArthur 9f356d6244 tokio-sync: add into_inner for TrySendErrors (#862) 2019-01-22 14:48:22 -08:00
Carl Lerche 13083153aa Introduce tokio-sync crate containing synchronization primitives. (#839)
Introduce a tokio-sync crate containing useful synchronization primitives for programs
written using Tokio.

The initial release contains:

* An mpsc channel
* A oneshot channel
* A semaphore implementation
* An `AtomicTask` primitive.

The `oneshot` and `mpsc` channels are new implementations providing improved
performance characteristics. In some benchmarks, the new mpsc channel shows
up to 7x improvement over the version provided by the `futures` crate. Unfortunately,
the `oneshot` implementation only provides a slight performance improvement as it
is mostly limited by the `futures` 0.1 task system. Once updated to the `std` version
of `Future` (currently nightly only), much greater performance improvements should
be achievable by `oneshot`.

Additionally, he implementations provided here are checked using
[Loom](http://github.com/carllerche/loom/), which provides greater confidence of
correctness.
2019-01-22 11:37:26 -08:00
rmcteggart-r7 91f20e33a4 docs: deal with Result instead of using unwrap (#860) 2019-01-20 14:21:17 -05:00
Eliza Weisman 983e9d1b67 timer: Fix DelayQueue delay reset logic (#851) 2019-01-20 08:38:39 -05:00
Stjepan Glavina 4c8f274db9 threadpool: drop incomplete tasks on shutdown (#722)
## Motivation

When the thread pool shuts down, futures that have been polled at least once but not completed yet are simply leaked. We should drop them instead.

## Solution

Multiple changes are introduced:

* Tasks are assigned a home worker the first time they are polled.

* Each worker contains a set of tasks (`Arc<Task>`) it is home to. When a task is assigned a home worker, it is registered in that worker's set of tasks. When the task is completed, it is unregistered from the set.

* When the thread pool shuts down and after all worker threads stop, the remaining tasks in workers' sets are aborted, i.e. they are switched to the `Aborted` state and their `Future`s are dropped.

* The thread pool shutdown process is refactored to make it more robust. We don't  track the number of active threads manually anymore. Instead, there's  `Arc<ShutdownTrigger>` that aborts remaining tasks and completes the `Shutdown` future once it gets destroyed (when all `Worker`s and `ThreadPool` get dropped because they're the only ones to contain strong references to the `ShutdownTrigger`).

Closes #424 
Closes #428
2019-01-17 22:12:25 +01:00
Marek Kotewicz c980837581 docs: missing links in tokio-timer::delay_queue (#845) 2019-01-13 21:20:08 +01:00
Marek Kotewicz eec370cae8 docs: fixed links in tokio-timer (#844)
* docs: fixed links in tokio-timer/src/timer/mod.rs

* docs: fixed links in tokio-timer::clock
2019-01-12 10:06:55 -08:00
Marek Kotewicz 733d432b80 docs: fixed links to tokio_timer::clock::Now (#842)
* docs: fixed links to tokio_timer::clock::Now in tokio-timer/src/timer/mod.rs

* docs: fixed links to std::time::Instant in tokio-timer/src/timer/mod.rs
2019-01-10 23:49:13 +01:00
Carl Lerche 74c473d68f travis: allow nightly Rust CI to fail (#843) 2019-01-10 11:27:58 -08:00
Sean McArthur d95c697781 tokio: update tokio-threadpool minimum version (#838) 2019-01-07 16:49:08 -08:00
Carl Lerche 25e835c5b7 tcp: specify version for tokio dev dependency
This is required for publishing to crates.io
2019-01-06 23:31:24 -08:00
Carl Lerche 961aae41c4 Bump version to 0.1.14. (#836)
Also bumps:

* tokio-async-await (0.1.5)
* tokio-executor (0.1.6)
* tokio-fs (0.1.5)
* tokio-io (0.1.11)
* tokio-reactor (0.1.8)
* tokio-tcp (0.1.3)
* tokio-threadpool (0.1.10)
* tokio-tls (0.2.1)
* tokio-uds (0.2.5)

...and updates LICENSE files to 2019.
2019-01-06 23:25:55 -08:00
Carl Lerche 74c73b218e Revert "util: implement stream debounce combinator (#747)" (#834)
This reverts commit 7a49ebb65e.

The commit conflicted with another change that was merged, causing CI to fail. The public API
also requires a bit more refinement (#833) and Tokio crates need to be released.
2019-01-06 16:56:49 -08:00
Moritz Gunz 7a49ebb65e util: implement stream debounce combinator (#747) 2019-01-05 11:08:12 -05:00
Stjepan Glavina a687922746 tcp: deprecate TcpStream::try_clone() (#824) 2019-01-05 10:55:58 -05:00
Stjepan Glavina df299ced45 threadpool: panic if a worker thread cannot be spawned (#826) 2019-01-05 10:53:38 -05:00
Carl Lerche 78d1fe0eb0 ci: limit min rust version to cargo check (#829) 2019-01-05 10:52:26 -05:00
Ryan Huang fc8cde383a docs: fix link to ThreadPool (#830) 2019-01-05 10:51:17 -05:00
Sean McArthur 76198f63d7 Provide optional features on tokio crate (#808)
Disabling all features means the only dependency is `futures`.

Relevant pieces of the API can then be enabled with the following features:

- `codec`
- `fs`
- `io`
- `reactor`
- `tcp`
- `timer`
- `udp`
- `uds`

This also introduces the beginnings of enabling only certain pieces of the `Runtime`. As a start, the entire default runtime API is enabled via the `rt-full` feature.
2019-01-04 11:42:33 -08:00
Carl Lerche 39dc5706b7 travis: remove commented out code. (#828)
The commented out lines are no longer relevant and will not be brought
back.
2019-01-03 22:04:09 -08:00
Carl Lerche cbecb87797 executor: fix build (#825)
Two unrelated PRs to the same file resulted in a broken build. This
patch fixes the build by including `Arc`.
2019-01-03 11:26:58 -08:00
Carl Lerche f0bdf1980c threadpool: remove unused fn (#822)
The unused lint on nightly has discovered a new unused fn.
2019-01-03 09:34:37 -08:00
Stjepan Glavina 5e2d93f060 Use Crossbeam's Parker/Unparker (#528) 2019-01-02 21:51:22 -08:00
Taiki Endo 9a8d087c69 Allow deprecated Error::cause (#818)
Error::cause is deprecated in Rust 1.33, but this allows Error::cause
until the minimum supported version of tokio is Rust 1.30.

When the minimum support version of tokio reaches Rust 1.30,
replace Error::cause with Error::source.

Fixes: #817
2019-01-02 14:12:11 -08:00
gralpli 30f59670c8 Clarify what NoopWaker does (#819) 2019-01-02 12:29:30 -08:00
jq-rs 9e4ddaeaf3 examples: single-threaded chat combinator example (#794) 2018-12-29 10:16:30 -05:00
Balthild Ires 03e2e864f3 Stablize pin feature (#814)
Box::pinned has been renamed to Box::pin. Meanwhile, the pin feature
no longer requires an attribute to enable.

Fixes: #813
2018-12-28 12:12:31 -08:00
Sean McArthur c8a990eda4 tokio-reactor: deprecates Handle::current() (#805)
The side effects of calling `Handle::current()` from outside of a
runtime could be very surprising, since it would start up a background
reactor.
2018-12-28 12:09:35 -08:00
Pavel Strakhov 1a5026324f executor: impl Unpark for Arc<Unpark> (#802) 2018-12-28 14:40:04 -05:00
Stjepan Glavina fdf4aba621 threadpool: introduce a global task queue (#798) 2018-12-28 14:34:54 -05:00
Roman 201b6ce53a ci: remove ALLOW_FAILURES=false in travis for nightly cargo doc (#816) 2018-12-28 10:06:00 -05:00
Roman db69275202 docs: fix warnings for nightly docs (#792) 2018-12-17 15:20:46 -05:00
Roman af85cb3430 ci: improve travis run times (#793) 2018-12-17 15:18:27 -05:00
Christian Bourjau 36f1a19ac8 Minor change in documentation of Decoder::decode (#797)
`None` -> `Ok(None)`
2018-12-13 11:14:38 -08:00
Stjepan Glavina 6aa990ea75 threadpool: fix semaphore deadlock (#795) 2018-12-12 16:42:18 -05:00
Simon Farnsworth 760a7667d6 threadpool: improve the documentation of blocking (#789) 2018-12-05 15:20:19 -05:00
Matt Gathu 2283b63e9e fs: added usage examples/doctests to File (#786) 2018-12-01 21:07:48 -05:00
Felix Obenhuber 8263e5f18d uds: fix WouldBlock case in UnixDatagram send methods (#782) 2018-11-30 19:40:03 -05:00
Carl Lerche b3e57b60d0 examples: remove reference to tokio-core (#780) 2018-11-28 14:55:31 -05:00
Matt Gathu 1cd0ebfc5e tcp: add usage examples to TcpListener and TcpStream (#775)
Refs: https://github.com/rust-lang-nursery/wg-net/issues/54
2018-11-28 13:05:35 -05:00
David Kellum 4797d79950 reactor: update to parking_lot 0.7 (#778) 2018-11-28 09:29:31 -08:00
Steven Fackler e7d9ba7e51 tls: make TlsConnector and TlsAcceptor derive Clone (#777) 2018-11-27 07:52:17 -05:00
luben karavelov 527dc0a66f net: export UnixDatagram and UnixDatagramFramed (#772) 2018-11-23 08:41:32 -05:00
Carl Lerche b117fc1d65 Bump version to v0.1.13 (#771)
This also bumps the following sub crate versions:

* tokio-current-thread (0.1.4)
* tokio-reactor (0.1.7)
* tokio-signal (0.2.7)
* tokio-threadpool (0.1.9)
* tokio-timer (0.2.8)
* tokio-udp (0.1.3)
* tokio-uds (0.2.4)
2018-11-21 17:11:31 -08:00
Felix Obenhuber 272e09d349 threadpool: remove smoke example (#764) (#770) 2018-11-21 14:23:36 -08:00
Stjepan Glavina 3235749006 threadpool: refactor pool shutdown (#769) 2018-11-20 21:43:23 +01:00
Stjepan Glavina 9c037044c4 threadpool: rename inner to something more descriptive (#768)
`inner` is a fitting name for variables of type named `Inner`, but in other cases I find them confusing - sometimes `inner` refers to a `Pool`, sometimes to a `Sender`. I renamed a bunch of variables named `inner` to be more descriptive.

This PR is the first step in an effort of splitting https://github.com/tokio-rs/tokio/pull/722#issuecomment-439552671 into multiple PRs.
2018-11-20 20:05:14 +01:00
Patrick Barrett 3658e10045 uds: implement UnixDatagramFramed (#453)
Implement `Stream + Sink` layer on top of unix domain sockets
using codecs.
2018-11-20 09:19:34 -08:00
Carl Lerche ed3ece266b current-thread: fix shutdown on idle (#763)
When spawning using `Handle` while on the executor, tasks were being
double counted. This prevented the number of active tasks to reach zero,
thus preventing the executor from shutting down.

This changes `spawn` to check if being called from the executor
**before** incrementing the number of active tasks.

Fixes #760
2018-11-20 09:17:07 -08:00
Liran Ringel 9b1a45cc6a tests: handle errors properly in examples (#748) 2018-11-20 11:10:36 -05:00
Carl Lerche 477fa5580a ci: Don't deploy docs if $TARGET is set (#762) 2018-11-19 21:22:28 -08:00
Toby Lawrence bb6cca8ff0 tests: switch to Windows Server 2016 for AppVeyor builds. (#761)
Should hopefully fix the underlying bug that was causing tokio-tls tests to occasionally fail on Windows.

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 20:18:37 -05:00
Moritz Gunz e166c4d912 Implement throttle combinator (#736)
Throttle down a stream by enforcing a fixed delay between items.
2018-11-19 15:04:55 -08:00
Toby Lawrence b7506cf663 Allow nightly builds to fail. (#743)
* tests: allow nightly builds to fail

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 17:13:56 -05:00
Carl Lerche dc4a29359f io: allow deprecated code in length_delimited test (#759)
This file is testing deprecated code, so it should be permitted to
access deprecated code.
2018-11-19 14:11:46 -08:00
Bastian Köcher d3dca4552b Expose after_start and before_stop in runtime::Builder (#756)
Closes #705
2018-11-19 09:04:58 -08:00
andoks 42a0df1ea4 Fix async await README example (#758)
* async-await: fix README example dependencies

As per commit "async-await: track nightly changes (#661)" ( commit
2f690d30bc)

> The `tokio-async-await` crate is no longer a facade. Instead, the
> `tokio` crate provides a feature flag to enable async/await support.

Ensure the example in the async-await README file also works by
correctly declaring this updated dependency

* async-await: remove unnecessary 'edition' declaration from README

As the "edition" feature was stabilized in rust v1.30 and async-await
specifies that the nightly toolchain must be used, remove the use of the
"edition" feature gate since it is enabled by default.
2018-11-17 20:58:19 -08:00
Brian Myers a98eab6eff rt: fix Builder docs to no longer use deprecated methods (#749) 2018-11-16 14:58:08 -08:00
Ivan Petkov 5a5dde70b3 signal: miscellaneous tweaks and improvements (#751)
* Minimize allocation needed for channels

* Use a newtype for signal ids

* We can just cast the raw pointer to a `usize` and still perform a
simple identity check, without incurring any implications of storing a
raw pointer (e.g. previously Signal was !Sync and had an unsafe impl of
Send, and now it is naturally Sync+Send)

* Broadcast with `try_send` instead of `start_send`

The `Stream::start_send` method uses backpressure and schedules the
current task to be notified whenever the channel has additional room,
which means we'll generate a lot of unnecessary wakeups whenever a
channel gets full

By changing to `try_send` and handling any errors, we ensure the
Driver's task won't get woken up when a Signal finally consumes its
notification, since we're coalescing things anyway
2018-11-16 14:56:43 -08:00
Alex Gaynor d0963774a3 chore: bump rand dependency to 0.6 (#753) 2018-11-16 14:54:14 -08:00
Felix Obenhuber c83355235c uds: minor doc fix in UnixStream and UnixDatagram (#754) 2018-11-16 14:53:19 -08:00
Kazuyoshi Kato 33a216e4c1 fs: add more tests (#755)
Fixes #704.
2018-11-16 14:50:06 -08:00
Toralf Wittner 09f2ac85bf udp: add into_parts to RecvDgram (#710)
* udp: add `into_parts` to `RecvDgram`

If `RecvDgram` can not be driven to completion it may become necessary to get back the `UdpSocket` it contains which is currently not possible.

This adds`into_parts` to get the socket as well as the buffer back. Both methods consume `RecvDgram`.

Note that after the future has completed, `into_parts` must not be used, or else a panic will happen.
2018-11-15 10:55:34 -05:00
Ohad Ravid 32a152630f uds: added solaris support in the ucred module (#733) 2018-11-15 10:30:37 -05:00
Kazuyoshi Kato 9153067d66 fs: add tests for directory-related functions (#704) (#724)
This change adds a few tests around directory-related functions.
2018-11-13 18:28:52 -05:00
Kazuyoshi Kato d246964bdf fs: gen_ascii_chars has been deprecated (#735)
Use sample_iter() instead.
2018-11-10 21:35:38 -05:00
Alex Gaynor e700607554 Bumped crossbeam-utils version (#746)
## Motivation

tokio depends on an out of date version of crossbeam-utils, which results in multiple versions of that package being linked in binaries which use other popular libraries.

## Solution

Bump the version; there's no API changes and tests still pass.
2018-11-10 10:39:09 +01:00
Benjamin Saunders 5321550534 Derive Clone for delay_queue::Key (#730)
Improves API ergonomics with minimal forwards-compatibility hazard.
2018-11-09 15:11:17 -08:00
Stjepan Glavina 32e1cafb57 fix tsan errors (#745) 2018-11-09 15:06:46 -08:00
Josh Leverette 49bc4025dd reactor: reduce log level of loop process (#734) 2018-11-07 16:53:53 -05:00
Carl Lerche 51e36e41bc Add tokio-buf and a BufStream trait (#611)
The `BufStream` trait provides an improved API for working with
asynchronous streams of bytes compared to `Stream<Item = [u8]>`
2018-10-29 13:43:48 -07:00
Carl Lerche d011b92b9a rt: fix Runtime::reactor() as used by tokio-core (#721)
* rt: fix `Runtime::reactor()` as used by tokio-core

Up until Tokio v0.1.11, the handle returned by `Runtime::reactor()`
pointed to a reactor instance running in a background thread. The thread
was eagerly spawned.

As of v0.1.12, a reactor instance is created per runtime worker thread.
`Runtime::reactor()` was deprecated and updated to point to the reactor
for one of the worker threads.

A problem occurs when attempting to use the reactor before spawning a
task. Worker threads are spawned lazily, which means that the reactor
referenced by `Runtime::reactor()` is not yet running.

This patch changes `Runtime::reactor` back to a dedicated reactor
running on a background thread. However, the background thread is now
spawned lazily when the deprecated function is first called.

Fixes #720

* Fix comment

Co-Authored-By: carllerche <[email protected]>
2018-10-25 11:23:54 +02:00
Carl Lerche f929576f0e Bump version to 0.1.12 (#718)
Also bumps the following sub-crates:

* tokio-fs (0.1.4)
* tokio-io (0.1.10)
* tokio-signal (0.2.6)
* tokio-threadpool (0.1.8)
* tokio-uds (0.2.3)
2018-10-23 22:00:49 -07:00
Ivan Petkov b0f001a05a signal: Bump version to 0.2.6 (#714)
* Also Update the CHANGELOG to match the rest of the project
2018-10-23 20:39:19 -07:00
Iku Iwasa 2291ba9d0d uds: add NetBSD support (#715) 2018-10-23 20:15:42 -04:00
Name 7f84f6b4ca contributing: fix an invalid link (#716)
Just move a dot to the right place.
2018-10-21 16:56:01 +00:00
Andrew Audibert 5f61bd5252 fix a typo in the contributing guide (#711) 2018-10-19 09:25:11 -07:00
Ryan Dahl bffa3ed558 fs: expose fs::File::from_std() (#696) 2018-10-17 19:51:46 -04:00
Sean McArthur 7b5ef61aeb runtime: check Enter in more places when blocking (#708)
- `tokio::run` checks Enter before creating a new threadpool and
  spawning the main future.
- `Runtime::block_on` now checks Enter
- `Runtime::block_on_all` now checks Enter
2018-10-17 15:25:40 -07:00
Stjepan Glavina 753336de8e threadpool: Arc instead of Inner in Notifier (#702) 2018-10-15 13:24:00 -07:00
Ryan Levick 65aea16ad1 tokio: change hello world to new, simpler example (#690) 2018-10-12 12:42:19 -04:00
nickelc 796fee6364 fs: fix minor documentation error for MetadataFuture (#698) 2018-10-12 12:41:19 -04:00
Stjepan Glavina adb0ba71d4 threadpool: worker threads shouldn't respect keep_alive (#692)
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.

Bug fixes and new features should include tests.

Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->

## Motivation

Now that each worker thread drives its own reactor, reactors have to be driven until the threadpool shuts down. We mustn't use the `keep_alive` setting to shut down a worker thread if it doesn't receive an event from the reactor for a certain duration of time.

<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->

## Solution

Just ignore the `keep_alive` setting when parking in `Worker::sleep`.

<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
2018-10-10 09:05:36 +02:00
David Ross bfa6766f3c re-export tokio_io::read in tokio::io (#689)
Fixes: #688
2018-10-09 19:50:03 -07:00
Nikolay Kim a2f457fa48 io: expose underlying codec (#686) 2018-10-06 19:23:05 -04:00
Eliza Weisman 1879bc49ce codec: Fix panic in LengthDelimitedCodec::encode (#682)
Fixes: #681 

## Motivation

Currently, a potential panic exists in `LengthDelimitedCodec::encode`.
Writing the length field to the `dst` buffer can exceed the buffer
capacity, as `BufMut::put_uint_{le,be}` doesn't reserve more capacity. 

## Solution

This branch adds a call to `dst.reserve` to ensure that there's 
sufficient remaining buffer capacity to hold the length field and
the frame, prior to writing the length field. Previously, capacity
was only reserved later in the function, when writing the frame
to the buffer, and we never reserved capacity for the length field.

I've also added a test that reproduces the issue. The test panics on
master, but passes after making this change.

Signed-off-by: Eliza Weisman <[email protected]>
2018-10-04 12:46:57 -07:00
Sven Marnach 678f6382b8 io: implement prepare_uninitialized_buffer for Take and Chain (#678) 2018-10-04 11:03:43 -07:00
Stjepan Glavina e27b0a46ba threadpool: spawn new tasks onto a random worker (#683)
* threadpool: submit new tasks to a random worker

* Revert unnecessary version bumps
2018-10-03 23:09:20 +02:00
Stjepan Glavina d35d0518f5 runtime: create reactor per worker (#660) 2018-10-02 18:19:27 -07:00
Sven Marnach 886511c0a6 io: fix minor documentation errors for Async{Read,Write} (#677) 2018-10-01 19:34:24 -04:00
Steven Fackler d06bd6b216 Expose keep_alive on the Runtime builder (#676)
This was overlooked when delegating the rest of the threadpool builder
methods from Runtime's builder.
2018-09-28 21:00:50 -07:00
Carl Lerche 2c85cd0991 Bump version to v0.1.11 (#675)
This fixes the dependency on `tokio-async-await` to not be scoped to
unix platforms.

Fixes #673
2018-09-28 11:32:52 -07:00
Carl Lerche 1e45237a28 Bump tokio-uds to v0.2.2 2018-09-27 20:05:23 -07:00
Sean McArthur 3a88d85538 ads: fix UdsStream::read_buf to clear read (not write) readiness (#672) 2018-09-27 20:02:29 -07:00
Carl Lerche b47ad24268 Bump version to v0.1.10, fixing minimal versions (#671)
Some minimal versions were not correctly updated.

Also updates:

* tokio-current-thread (v0.1.3).
2018-09-27 13:00:53 -07:00
Carl Lerche cab9a44e01 Bump version to v0.1.9 (#666)
This also includes bumps to subcrates.

* tokio-async-await (0.1.4)
* tokio-codec (0.1.1)
* tokio-current-thread (0.1.2)
* tokio-executor (0.1.5)
* tokio-io (0.1.9)
* tokio-reactor (0.1.6)
* tokio-tcp (0.1.2)
* tokio-threadpool (0.1.7)
* tokio-timer (0.2.7)
2018-09-26 22:32:51 -07:00
Carl Lerche 964afb2ce3 split facade modules into separate files (#665) 2018-09-26 15:28:57 -07:00
Carl Lerche 2f690d30bc async-await: track nightly changes (#661)
The `tokio-async-await` crate is no longer a facade. Instead, the `tokio` crate
provides a feature flag to enable async/await support.
2018-09-26 10:10:47 -07:00
Stjepan Glavina 331a88cee6 reactor: turn bench-poll into a proper benchmark (#662) 2018-09-26 08:52:35 -07:00
Andrew Tunnell-Jones 46353737e7 tinydb: Update doc to reflect change from RefCell to Mutex (#663)
Fixes: #658
2018-09-26 11:47:31 +02:00
Carl Lerche ffd73a64e7 async-await: streaming hyper body example (#656)
Add reading the hyper body in the async/await example.
2018-09-21 19:58:46 -07:00
Toby Lawrence 1119d572ee io: ensure ReadHalf/WriteHalf do not return WouldBlock directly (#655)
* io: ensure ReadHalf/WriteHalf do not return WouldBlock directly

These facades were passing back WouldBlock when the internal BiLock
couldn't be acquired, which does not fit the intended behavior.

Signed-off-by: Toby Lawrence <[email protected]>

* io: pull from the local crate, not crates.io
2018-09-21 14:59:07 -04:00
Stjepan Glavina 20ca59114a threadpool: impl Drop for Queue (#649)
We need to drain the queue when dropping, or else those `Arc<Task>`s
will be leaked.

Fixes #542
2018-09-21 10:20:41 -07:00
Eliza Weisman 3dd95a9ff1 Add max line length to LinesCodec (#632)
## Motivation

Currently, there is a potential denial of service vulnerability in the
`lines` codec. Since there is no bound on the buffer that holds data
before it is split into a new line, an attacker could send an unbounded
amount of data without sending a `\n` character. 

## Solution

This branch adds a `new_with_max_length` constructor for `LinesCodec`
that configures a limit on the maximum number of bytes per line. When
the limit is reached, the the overly long line will be discarded (in 
`max_length`-sized increments until a newline character or the end of the
buffer is reached. It was also necessary to add some special-case logic
to avoid creating an empty line when the length limit is reached at the 
character immediately _before_ a `\n` character.

Additionally, this branch adds new tests for this function, including a
test for changing the line limit in-flight.

## Notes

This branch makes the following changes from my original PR with
this change (#590):

- The whole too-long line is discarded at once in the first call to `decode`
  that encounters it.
- Only one error is emitted per too-long line.
- Made all the changes requested by @carllerche in
  https://github.com/tokio-rs/tokio/pull/590#issuecomment-420735023

Fixes: #186 

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-20 17:08:00 -07:00
Sven Marnach be67eda117 fix deprecation warning in test for FutureExt::deadline() (#651)
* silence deprecation warnings for deadline in tests
* add new integration test for timeout
2018-09-20 15:26:56 -07:00
Alexander Polakov e267a1922d Reexport TaskExecutor from tokio_current_thread (#652) 2018-09-19 14:46:13 -07:00
Eliza Weisman 9b456f48d9 Set RUST_BACKTRACE=1 on AppVeyor (#650)
## Motivation

Currently, the `RUST_BACKTRACE` environment variable is set to `1` on
Travis CI builds:
https://github.com/tokio-rs/tokio/blob/0ca973a7ebc5b8a29beac1ccb6c73ef26ddcbf22/.travis.yml#L49
However, it's not set on AppVeyor. This can make debugging
Windows-specific CI failures challenging for developers on other
operating systems.

## Solution

This branch sets `RUST_BACKTRACE=1` on AppVeyor.

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-19 11:53:11 -07:00
Eliza Weisman 0ca973a7eb tokio: deprecate and replace runtime::threadpool_builder (#645)
* Deprecate and hide runtime::Builder::threadpool_builder

* Add functions to runtime::Builder wrapping threadpool builder functions

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-19 10:37:45 -04:00
RT df6acf0c2a tokio-timer: reset timeout after elapsed in stream (#648) 2018-09-19 09:47:39 -04:00
Eliza Weisman 98d23b8b29 Make tokio::run panic if called from inside tokio::run (#646)
This is implemented by creating an `Enter` instance from within `run`.

This patch also introduces `Enter::block_on`.

Fixes #504
2018-09-18 21:57:21 -07:00
Eliza Weisman 85f8522536 tokio-executor: hide deprecated tokio-threadpool reexports (#644)
Fixes: #643
Signed-off-by: Eliza Weisman <[email protected]>
2018-09-18 23:57:17 -04:00
Liran Ringel d275341fb2 Fix tokio-async-await tests compile errors (#630) 2018-09-18 13:49:08 -07:00
Nick Cameron d735e5d527 async-await: update deps in tokio-async-await (#639) 2018-09-18 10:08:35 -07:00
Carl Lerche 4019198706 Add some missing future::Executor implementations (#563)
This adds an implementation of future::Executor for
`executor::DefaultExecutor` and `runtime::current_thread::Handle`.
2018-09-17 22:23:48 -07:00
Ivan Petkov 24dc85dc5e ci: Run cargo test with the --no-fail-fast flag (#635)
Since the CI runs all tests for all tokio crates, it is possible that a
sporadic failure in one crate can mask failures/successes of other
crates' tests.

Using the `--no-fail-fast` flag instructs cargo to run *all* tests
before failing the build. This will allow checking to see if any
relevant test cases still pass even if an unrelated test has failed.
2018-09-15 00:41:41 +00:00
Ivan Petkov aaa5adb7fd Merge pull request #634 from vorner/import-signal-2
Import the `tokio-signal` source from its original repo

Original repository can be found at https://github.com/alexcrichton/tokio-signal
2018-09-14 22:58:19 +00:00
Michal 'vorner' Vaner 5f68b3aaa1 signal: Remove Apache license
Whole tokio is MIT only, unifying.
2018-09-14 23:28:56 +02:00
Michal 'vorner' Vaner 2f69acbe9f signal: Fix tests after importing & linking
* Don't use tokio-core any more for tests. That one brings tokio from
  crates.io instead of the current workspace and two versions of that
  don't want to cooperate.
* Guard unix-specific examples on windows.
* Leave CI setup to top-level directory.
2018-09-14 23:28:47 +02:00
Michal 'vorner' Vaner 7e12f5c39e signal: Link tokio-signal and tokio crates
References in the Cargo.toml, various links.
2018-09-14 23:28:33 +02:00
Michal 'vorner' Vaner 462882b356 Merge tokio with tokio-signal 2018-09-14 23:27:22 +02:00
Michal 'vorner' Vaner 35687f1d18 signal: Move to tokio-signal subdirectory
As a preparation to merge with tokio.
2018-09-14 23:25:57 +02:00
Michal 'vorner' Vaner e7dc3a1091 signal: Use signal-hook for registration of signals
This saves some code and gets rid of quite some amount of unsafe code.
2018-09-14 23:25:21 +02:00
Ivan Petkov b594e240f9 signal: Bump version to 0.2.5 2018-09-14 23:25:21 +02:00
Ivan Petkov 605708dca6 signal: Fix a possible starvation with concurrent Signal polls
* Originally reported in alexcrichton/tokio-process#42
* The root cause appears to be due to two different PollEvented
instances trying to consume readiness events from the same file
descriptor.
* Previously we would simply swallow any `AlreadyExists` errors when
attempting to register the pipe receiver with the event loop. I'm not
sure if this means the PollEvented wrapper wasn't fully registered to
receive events, or maybe there is a potential race condition with how
PollEvented consumes mio readiness events. Using a fresh/duplicate file
descriptor appears to mitigate the issue, however.
* I was also not able to reproduce the issue as an isolated test case so
there is no regression test available within this crate (but we can add
one in tokio-process)
2018-09-14 23:25:08 +02:00
Carl Lerche cc40a4e7f0 Revert "Add max line length to LinesCodec (#590)"
This reverts commit 4ae6c997ee.
2018-09-12 10:34:37 -07:00
Eliza Weisman 4ae6c997ee Add max line length to LinesCodec (#590)
* codec: add new constructor `with_max_length ` to `LinesCodec`
* codec: add security note to docs

Signed-off-by: Eliza Weisman <[email protected]>

* Fix Rust 1.25 compatibility

* codec: Fix incorrect line lengths in tests (and add assertions)

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix off-by-one error in lines codec

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix call to decode rather than decode_eof in test

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix incorrect LinesCodec::decode_max_line_length

This bug was introduced after the fix for the off-by-one error.
Fortunately, the doctests caught it.

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Minor style improvements

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Don't allow LinesCodec length limit to be set after construction

Signed-off-by: Eliza Weisman <[email protected]>

* codec: change LinesCodec to error and discard line when at max length

* codec: Fix build on Rust 1.25

The slice patterns syntax wasn't supported yet in that release.

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Add test for out-of-bounds index when peeking

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix out of bounds index

* codec: Fix incomplete comment

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Add test for line decoder buffer underrun
2018-09-12 13:02:57 -04:00
Stjepan Glavina 0f44adf5f6 reactor: use LocalKey::try_with in sharded RW lock (#628)
@jonhoo reported a panic in the call to `LocalKey::with`, which occurs
when the reactor is dropped in the middle of TLS teardown. This PR
changes the call to `LocalKey::try_with` and handles the case when the
thread-local value has already been destroyed.
2018-09-11 16:48:06 -04:00
Flux Xu 19d5565442 Expose thread_pool::SpawnHandle (#604) 2018-09-10 15:51:15 -07:00
Ivan Petkov 98e76d9bc6 signal: Bump version to to 0.2.4 2018-09-10 11:30:08 -07:00
Alan Somers b7f5bc95fe signal: Actually make unix::bsd public 2018-09-10 11:30:07 -07:00
Ivan Petkov 90ea2f6c5b signal: Bump version to 0.2.3 2018-09-10 11:30:07 -07:00
Ivan Petkov 837c3934d5 signal: Also cfg gate the entire unix::bsd module 2018-09-10 11:30:07 -07:00
Alan Somers bcd42d11d9 signal: Move SIGINFO to a BSD-specific submodule 2018-09-10 11:30:07 -07:00
Alan Somers 8ad66d296f signal: Add CHANGELOG entry for SIGINFO. 2018-09-10 11:30:07 -07:00
Alan Somers d9edc26e97 signal: export SIGINFO on supported platforms. 2018-09-10 11:30:07 -07:00
Alan Somers f0ac62151b signal: Update tokio dependency
tokio::runtime::current_thread was added in 0.1.6
2018-09-10 11:30:06 -07:00
Ivan Petkov 214722a296 signal: Update CHANGELOG 2018-09-10 11:30:06 -07:00
Niv Kaminer b8f8145b62 signal: cast SIGINFO to be the same type as sa_flags regardless of platform 2018-09-10 11:30:06 -07:00
Ivan Petkov 32d3e0e1f9 signal: Bump version to 0.2.2 2018-09-10 11:30:06 -07:00
Ivan Petkov 266919add6 signal: Refactor Signal tests
* Added timeouts to all tests that were missing them
 - any issue we have will likely result in deadlocks/starvation so its
   best if all tests quickly timeout rather than require getting killed
   or have the CI timeout itself
* Added a `support` module and put a bunch of helpers there to DRY the
tests
2018-09-10 11:30:06 -07:00
Ivan Petkov c9ffd98b1e signal: Fix a potential Signal starvation based on creation order
* As observed in alexcrichton/tokio-signal#38, Signal instances can starve based on the order
they are created in, and this ordering appears to be platform/OS
specific
* The crux of the issue is that we woud only *attempt* to broadcast any
pending signals if we successfully read out at least one byte from the
global pipe.
* For reasons unclear to me, the affected Signal instance would get
woken up after the signal handler writes to the global pipe, but it
would immediately hit a WouldBlock error and give up, bypassing the
broadcast attempt (even though the pending flag was correctly set).
 - Maybe this has to do with OS specifics with how the bytes are
   delivered (or not), or with some complex interaction with tokio and
   the pipe registration. It seems fishy since strace logs didn't show
   the signal handler pipe write fail either, but I'm all out of ideas
* The fix appears simple: unconditionally attempt to broadcast any
pending signals *any* time a Driver instance is woken up.
* Since we perform an atomic check for each pending signal, we know that
each (coalesced) signal broadcast will happen at most once. If we were
supuriously woken up and no signals were pending, then nothing will be
yielded to any pollers of Signal
* The down side is that since each Signal instance polls a Driver
instance, each poll to Signal will essentially perform N atomic
operations (N = number of signals we support) in an attempt to broadcast
any pending signals.
 - However, we can revisit optimizing this better in the future

Fixes alexcrichton/tokio-signal#38
2018-09-10 11:30:06 -07:00
Ivan Petkov 2d4bfa1485 signal: Fix starvation of signal streams on drop of another instance
* We introduce a new global structure which keeps track of how many
signal streams have been registered with a given event loop (the event
loop is identified by its OS file descriptor)
* We only attempt to deregister our global evented pipe from any event
loop if and only if we are the last signal that was registered with it
2018-09-10 11:30:05 -07:00
Ivan Petkov b6ecfa251c signal: Add (failing) test case which exibits starvation on drop
* Currently, whenever a new signal stream is created we attempt to
register a global pipe with the event loop to drive events.
* We also (correctly) swallow any descriptor-already-registered errors
since the same pipe is always used
* However, we currently *deregister* the same global pipe *any time* a
Signal stream is dropped.
* This means that if 2 or more of Signal instances exist simultaneously
(even if listening for different signals) and one of them is dropped,
the remainder will starve (until any new signal is created again).
2018-09-10 11:30:05 -07:00
Ivan Petkov 3a81d7746a signal: Split up all integration tests to run in their own process
* Cargo runs each integration-style-test in its own process. Since the
tests use global data structures specific to the process, we should run
them in an isolated manner to avoid having cross-test interactions
* Fixes alexcrichton/tokio-signal#39
2018-09-10 11:30:05 -07:00
Daniel Wagner-Hall 6e1a833825 signal: Update mio dependency to 0.6.14
This allows tokio-signal to build with `-Z minimal-versions` - see
https://github.com/rust-lang/cargo/issues/5657#issuecomment-401110172
for more details.

Earlier versions depend on log 0.3.1, which itself depends on libc
0.1, which doesn't build on any post-1.0 version of rust.
2018-09-10 11:30:05 -07:00
Niv Kaminer 3f80953dee signal: account for definition mismatch on aarch64 android 2018-09-10 11:30:05 -07:00
Michael Hadley 4374f5be70 signal: Fix typo in README.md 2018-09-10 11:30:05 -07:00
jjl b23ab94cd5 signal: Change reference from 'tokio-core' to 'tokio' in README 2018-09-10 11:30:04 -07:00
Markus Westerlind 40c77bd17e signal: Version 0.2 2018-09-10 11:30:04 -07:00
Markus Westerlind 31b51004f2 signal: Increase number of signals to 33 for the sake of FreeBSD
Fixes alexcrichton/tokio-signal#21
2018-09-10 11:30:04 -07:00
Markus Westerlind 9a4e4f2308 signal: Don't use the depreceated new method of Registration 2018-09-10 11:30:04 -07:00
Markus Westerlind 484fda7a23 signal: Add an appveyor build file
It is not possible to test much on windows but this will at least verify
that it can be built
2018-09-10 11:30:04 -07:00
Markus Westerlind 45ba6e2652 signal: Remove test that were accidentally included in the ctrl-c example 2018-09-10 11:30:04 -07:00
Markus Westerlind e73b8a0cc9 signal: refactor: Prefer the implicit handle passing used by tokio 2018-09-10 11:30:03 -07:00
Markus Westerlind 209232befd signal: Ensure that the driver dies once the signal does
`tokio::run` expects that all futures finish processing so we can't
leave `Driver` around forever or `tokio::run` would never return.
2018-09-10 11:30:03 -07:00
Markus Westerlind f759e4d70f signal: panic 2018-09-10 11:30:03 -07:00
Markus Westerlind 1fdff707b8 signal: test: Add a test for ctrl_c on unix 2018-09-10 11:30:03 -07:00
Markus Westerlind e97e8cb7fe signal: Update the windows implementation to work with tokio
BREAKING CHANGE

`ctrl_c` now takes a `tokio_reactor::Handle`
2018-09-10 11:30:03 -07:00
Markus Westerlind 2848df9b6c signal: Run rustfmt 0.4.2 2018-09-10 11:30:03 -07:00
Alex Crichton 7a24ed7509 signal: Bump to 0.1.5 2018-09-10 11:30:02 -07:00
Alex Crichton 9c9760cfbb signal: Fix a bug with most recent tokio-core release 2018-09-10 11:30:02 -07:00
Alex Crichton 5ecd929b1a signal: Bump to 0.1.4 2018-09-10 11:30:02 -07:00
Alex Crichton 8ddebf4309 signal: Fix compile on Android
Closes alexcrichton/tokio-signal#19
2018-09-10 11:30:02 -07:00
Alex Crichton 0df1882f21 signal: Bump to 0.1.3 2018-09-10 11:30:02 -07:00
Alex Crichton 4fa1b2b58c signal: Update to winapi 0.3 2018-09-10 11:30:02 -07:00
Alex Crichton a4895fe364 signal: Tweak travis config 2018-09-10 11:30:02 -07:00
Alex Crichton 7ab97f99c8 signal: Clarify wording of license information in README.
This text historically was copied verbatim from rust-lang/rust's own README [1]
with the intention of licensing projects the same as rustc's own license, namely
a dual MIT/Apache-2.0 license. The clause about "various BSD-like licenses"
isn't actually correct for almost all projects other than rust-lang/rust and
the wording around "both" was slightly ambiguous.

This commit updates the wording to match more precisely what's in the
standard library [2], namely clarifying that there aren't any BSD-like licenses
in this repository and that the source is licensable under either license, at
your own discretion.

[1]: https://github.com/rust-lang/rust/tree/f0fe716dbcbf2363ab8f929325d32a17e51039d0#license
[2]: https://github.com/rust-lang/rust/blob/f0fe716dbcbf2363ab8f929325d32a17e51039d0/src/libstd/lib.rs#L5-L9
2018-09-10 11:30:01 -07:00
Alex Crichton 9bf3228f73 signal: Add an example for waiting on two signals
Relies on `Stream::select` to merge streams.

Closes alexcrichton/tokio-signal#16
2018-09-10 11:30:01 -07:00
Raphael Nestler d41c60e21d signal: Fix typo in README 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 934c596133 signal: Ctrl+C example: quit after 10 signals. 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 72e2209bd8 signal: Ctrl+C example: more explanations 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 6a7092b9f7 signal: Ctrl+C example: Defer stream initialisation (and explain how/why) 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 1c893ef6d3 signal: undo nested example cargo project
.. after learning about `cargo run --example`
2018-09-10 11:30:00 -07:00
Jules Kerssemakers 20e7598e8d signal: Ctrl+C example: Don't forget proper attribution for original example 2018-09-10 11:30:00 -07:00
Jules Kerssemakers 3db92496f6 signal: Ctrl+C example: highlight power of Stream::for_each() 2018-09-10 11:30:00 -07:00
Jules Kerssemakers da47cfbd58 signal: Ctrl+C example: clarify control flow after receiving Ctrl+C: unreachable!() 2018-09-10 11:30:00 -07:00
Jules Kerssemakers f5eadc74f1 signal: Ctrl+C example: add explanatory comments 2018-09-10 11:30:00 -07:00
Jules Kerssemakers 175f9afea9 signal: ctrl+C example: Prompt user to do something.
So we don't stay at a blank terminal without any feedback after `cargo run`
2018-09-10 11:30:00 -07:00
Jules Kerssemakers 48eda3fe2f signal: Upgrade ctrl+c example into cargo run-able version with proper Cargo.toml 2018-09-10 11:29:59 -07:00
Alex Crichton 010c2223ca signal: Touch up the sighup-example slightly 2018-09-10 11:29:59 -07:00
Jules Kerssemakers 36b58d8fa8 signal: new example: SIGHUP, shows how to receive other signals than ctrl+C 2018-09-10 11:29:59 -07:00
Alex Crichton c601f68c9f signal: Add some examples to crate docs
Closes alexcrichton/tokio-signal#11
2018-09-10 11:29:59 -07:00
Alex Crichton 78ca103f3a signal: Update to tokio-io 2018-09-10 11:29:59 -07:00
Michal 'vorner' Vaner 7da00f3832 signal: Use IDs that don't run out
Replace the sequential counting (which might be exhausted) by an address
of an object (in a box, so it doesn't change). This is also a unique, so
it is acceptable ID.
2018-09-10 11:29:59 -07:00
Michal 'vorner' Vaner edba77e8df signal: A test running multiple event loops
Run multiple loops (both in parallel and sequentially) to make sure
broadcasting to multiple of them works and we work even after the
initial loop has gone away.
2018-09-10 11:29:58 -07:00
Michal 'vorner' Vaner cf1afd2d90 signal: Style: Replace tabs with spaces
Mixing tabs and spaces breaks indentation for people (and github) if
they use different tab width.
2018-09-10 11:29:58 -07:00
Alex Crichton 8e58a9d8d4 signal: Add badges/categories 2018-09-10 11:29:58 -07:00
Alex Crichton 4afef9391a signal: Bump to 0.1.2 2018-09-10 11:29:58 -07:00
Alex Crichton b6bacc1ca3 signal: Clarify a comment 2018-09-10 11:29:58 -07:00
Alex Crichton 955cd2836d signal: Clear out old Signal on drop 2018-09-10 11:29:58 -07:00
Alex Crichton 70e4ed67ad signal: Touch up more impls and comments 2018-09-10 11:29:58 -07:00
Alex Crichton 699b9ab89e signal: Handle a few more errors 2018-09-10 11:29:57 -07:00
Alex Crichton ba0921a01d signal: Various cleanups:
* Drop nix/lazy_static
* Use previously registered handlers
* Handle some more errors
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 367cb56e02 signal: The driver task
Add the driver task, connecting the signal handler wakeups to the
wakeups of of the streams.

It is a prototype-quality code, a lot of cleanups and similar is needed.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 04d949c380 signal: Provide the new kind of Signal stream
Which is just a wrapper around the futures::sync::mpsc. The sender is in
a global registry.

The part that connects the wakeups to the senders in the registry
doesn't yet exist.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner ed4359bb26 signal: Implement the wake-up part of the new signal handling
Register the signal handler that wakes up someone through a self-pipe.
That someone doesn't yet exist, though.

Some dependencies (nix, lazy_static) added to speed up the prototyping
process. They are likely to be dropped in some future commits.

Some features (eg. preserving the previous signal handlers) are still
missing.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 4142dc2fae signal: Use tokio-core from git
Just for now, as we need some yet unreleased features.
2018-09-10 11:29:57 -07:00
Alex Crichton 468b037e4e signal: Update docs urls and such 2018-09-10 11:29:56 -07:00
Alex Crichton b33ae3cdd6 signal: Remove deprecated API usage on Windows 2018-09-10 11:29:56 -07:00
Alex Crichton 4519ac8e17 signal: Remove use of deprecated APIs on Unix 2018-09-10 11:29:56 -07:00
Alex Crichton 92b93ee176 signal: Bump to 0.1.1 2018-09-10 11:29:56 -07:00
Alex Crichton 635149e3ab signal: Ignore errors in signal handler
Closes alexcrichton/tokio-signal#3
2018-09-10 11:29:56 -07:00
Alex Crichton e28c350e31 signal: Update travis token 2018-09-10 11:29:56 -07:00
Chris Emerson 1a122018a2 signal: Trivial typo fix. 2018-09-10 11:29:55 -07:00
Alex Crichton f3f8ee431e signal: Remove SIGKILL reexport 2018-09-10 11:29:55 -07:00
Alex Crichton 61c4047c6a signal: Add symbolic reexports for common signals
Means you don't have to import libc!

Closes alexcrichton/tokio-signal#1
2018-09-10 11:29:55 -07:00
Alex Crichton 3486a61a0f signal: Update deps to point to crates.io 2018-09-10 11:29:55 -07:00
Alex Crichton 8291c3d462 signal: Start adding windows support 2018-09-10 11:29:55 -07:00
Alex Crichton 1b6893b6f6 signal: Track tokio-core master 2018-09-10 11:29:55 -07:00
Alex Crichton eca7f0760f signal: Initial commit 2018-09-10 11:29:50 -07:00
Ben Boeckel 89d969d518 StreamExt: add a trait for additional Stream methods (#573)
Primarily, it offers a `timeout` method for streams.
2018-09-07 15:43:03 -07:00
Carl Lerche 16664189c1 async-await: move examples into dedicated crate (#608)
This works around a bug in the cargo renaming feature as well as allows
the use of `[patch]` in the `Cargo.toml`.
2018-09-06 13:36:59 -04:00
Carl Lerche 6828870608 async-await: bump version to v0.1.2 (#619) 2018-09-04 15:02:15 -07:00
Nimi Wariboko Jr 89d0cda2e2 async-await: use new PinMut/PinBox location (#613)
The types moved in `std`. This patch updates tokio-async-await to
import `PinMut` and `PinBox` from the new location.

Ref: rust-lang/rust#53227
2018-09-04 13:19:10 -07:00
Stjepan Glavina 8052a9b348 guide: fix a few typos (#612) 2018-09-03 10:18:53 -07:00
ksqsf a5ac6c8b72 Fix undesired multi-line error message (#605) 2018-08-31 09:53:41 -07:00
Léo Gaspard 3a59526523 Document that timeout-ed futures will be polled at least once (#603)
tokio-util: document behavior of `StreamExt::timeout` when timeout = 0
2018-08-31 09:26:41 -04:00
Zachary Stewart 322a94f72f Update documentation for AsyncRead and AsyncWrite (#596)
tokio-io: update documentation for AsyncRead and AsyncWrite
2018-08-31 09:00:32 -04:00
Eunchong Yu c03b23355b Fix minimum version to export tokio::codec module (#594)
tokio-async-await: fix minimum version to export tokio::codec module (#594)
2018-08-31 08:02:02 -04:00
Eliza Weisman bc91bc5022 Fix non-terminating loop in tokio_io::length_delimited::FramedWrite (#576)
* tokio-io: fix non-terminating loop in length_delimited::FramedWrite (#497)
2018-08-31 06:31:43 -04:00
Flux Xu a7b053372f Add ThreadPool::spawn_handle (#602)
## Motivation

`tokio_threadpool::ThreadPool::spawn` has no return value.

## Solution

Add `ThreadPool::spawn_handle` which calls
`futures::sync::oneshot::spawn` to return a future represents the return
value.
2018-08-30 16:53:05 -07:00
Eliza Weisman 673fdb5cb3 Refactor codec::length_delimited (#575)
This patch refactors `length_delimited` to be implemented as a `Codec` and
use the default `Framed` wrapper types.

The original implementation did not do this in order to support vectored writes in the
write half. However, this implementation would be more efficient with small frames anyway.

If vectored writes are to be explored in the future, then it should be done holistically.

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-30 14:50:32 -07:00
Carl Lerche 97618746de readme: fix section ordering (#600) 2018-08-30 14:46:40 -07:00
Carl Lerche d8f8b59df9 guide: add a testing section to the contributing guide (#598) 2018-08-30 12:26:24 -07:00
Jon Gjengset 0745a9b88a Use spawn_local to spawn from local Handles (#565)
Previously, every call to `current_thread::Handle::spawn` would go
through a `mpsc` channel. This is unnecessary when the `Handle` is still
on the same thread as the current thread executor. This patch fixes that
by storing the `ThreadId` of the executor when it is created, and then
comparing against that when `Handle::spawn` is called. If the call is
made from the same thread, `spawn_local` is used directly.

Fixes #562.
2018-08-30 11:24:59 -07:00
Eliza Weisman a7f5ba28ba Bump minimum supported version & document support policy (#599)
* Bump minimum supported version & document support policy

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-29 20:35:27 -04:00
Josef Brandl cc3b6af7a3 Fix tokio-uds version (#580) 2018-08-28 15:12:45 -07:00
Martin Chaine 07e30ae923 net: rework tokio_tcp and tokio_udp re-exports (#548)
This patch keeps the primary net types in `tokio::net` and moves
secondary types to a protocol specific submodules.

Primary types are the ones that users are most likely to name (`TcpStream`,
`TcpListener`, `UdpSocket`, ...)

Secondary types are the operation futures.
2018-08-28 14:13:06 -07:00
Jason Davies 69d90ac7ee Fix a few typos in timer docs. (#569) 2018-08-28 11:00:42 -07:00
Carl Lerche d16032cf06 async-await: misc fixes and typos (#585) 2018-08-27 15:16:32 -07:00
Carl Lerche b479ce78d3 add experimental async/await support. (#582)
This patch adds experimental async/await support to Tokio. It does this
by adding feature flags to existing libs only where necessary in order
to add nightly specific code (mostly `Unpin` implementations). It then
provides a new crate: `tokio-async-await` which is a shim layer on top
of `tokio`.

The `tokio-async-await` crate is expected to look exactly like `tokio`
does, but with async / await support. This strategy reduces the amount
of cfg guarding in the main libraries.

This patch also adds `tokio-channel`, which is copied from futures-rs
0.1 and adds the necessary `Unpin` implementations. In general, futures
0.1 is mostly unmaintained, so it will make sense for Tokio to take over
maintainership of key components regardless of async / await support.
2018-08-27 12:24:51 -07:00
Michal 'vorner' Vaner 6e45e0ac61 re-export tokio-current-thread::spawn (#579)
Re-export it inside the tokio::runtime::current_thread, as the original
place (tokio::executor::current_thread) is hidden from documentation and
users need some way to spawn non-Send futures.
2018-08-25 12:51:49 -07:00
Ben Boeckel 82c5baa09b Spelling fixes (#571)
* docs: fix spelling and whitespace errors
2018-08-25 15:26:41 -04:00
Carl Lerche 7dc6404726 draft initial CONTRIBUTING guide (#567)
This guide was adopted from the node.js project.
2018-08-24 13:03:34 -07:00
Eliza Weisman 2e88e29fe9 Move tokio_io::codec::length_delimited module to tokio::codec (#568)
* Deprecate tokio-io::length_delimited
* Move `length_delimited` into `tokio::codec`

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-24 15:54:42 -04:00
Carl Lerche 07203408de Bump version to v0.1.8 (#566)
This also bumps a number of sub crates:

* tokio-executor (0.1.3)
* tokio-io (0.1.8)
* tokio-reactor (0.1.4)
* tokio-threadpool (0.1.6)
* tokio-timer (0.2.6)
* tokio-udp (0.1.2)
2018-08-24 08:58:26 -07:00
Carl Lerche 8bf2e9aeb0 Introduce Timeout and deprecate Deadline. (#558)
This patch introduces `Timeout`. This new type allows setting a timeout
both using a duration and an instant. Given this overlap with
`Deadline`, `Deadline` is deprecated.

In addition to supporting future timeouts, the `Timeout` combinator is
able to provide timeout functionality to streams. It does this by
applying a duration based timeout to each item being yielded.

The main reason for introducing `Timeout` is that a deadline approach
does not work with streams. Since `Timeout` needed to be introduced
anyway, keeping `Deadline` around does not make sense.
2018-08-22 20:39:46 -07:00
Carl Lerche cf184eb326 timer: Reduce size of Delay struct (#554)
* Remove `counted` field on `timer::Entry`.

It turns out that a better indicator of whether or not the number of
active timeouts should be decremented is if the `Entry` has been
associated with a timer. In other words, if `Entry::inner` can be
upgraded, then the count should be decremented on drop.

* timer: Tweak link between `Delay` and the driver

This tweaks the struct layout / details regarding how a `Delay` instance
is linked to a driver (timer instance). Instead of lazily allocating the
`Entry` (node shared between `Delay` and the timer), `Entry` is
allocated immediately when `Delay` is created. This allows using the
entry store data used by `Delay`.

This is in anticipation of further timer improvements that would
otherwise require the size of `Delay` to grow further. Since an
allocation is already made, the idea is to shrink the size of the
`Delay` struct.
2018-08-21 21:48:40 -07:00
Carl Lerche d822b721b4 Add DelayQueue implementation to tokio-timer (#550)
This patch adds a `DelayQueue` to tokio_timer. The `DelayQueue` allows
inserting elements as well as specifying a time at which the element
should be returned to the user. This allows handling more complex
timeout situations.
2018-08-20 21:47:10 -07:00
Carl Lerche c66b56c3fb Implement Default for tokio_timer::Handle (#553)
This patch implements `Default` for `tokio_timer::Handle`. It returns a
`Handle` instance that is not bound to a specific timer. Instead, it
will use the timer for the current execution context. This is the same
strategy used by `tokio_reactor::Handle`.

Fixes #547
2018-08-20 13:01:39 -07:00
Carl Lerche 89639ec48b Bump tokio-uds version to v0.2.1 (#552)
Fixes #551
2018-08-19 21:23:49 -07:00
Martin Chaine 2b1b0ac858 Expose tokio_uds from the root crate (#526) 2018-08-15 21:26:10 -07:00
Carl Lerche 6b84c73f12 tokio::codec docs + additional exports (#546) 2018-08-15 21:25:25 -07:00
Jason Ish 767b370c21 Add tokio-tls echo example. (#541)
Based on the current example on the front page of tokio.rs.
2018-08-15 07:54:24 -04:00
Gary M. Josack 28010b5962 Update lines_encoder test to use LinesCodec (#544)
The `lines_encoder` test is a copy/paste of `bytes_encoder` and not
testing the LinesCodec encoding at all. This updates the test to do
simple validations of the LinesCodec encoding.
2018-08-15 07:51:42 -04:00
Roman 2e343f9e42 Reexport Encoder, Decoder, Framed* from tokio::codec (#499) 2018-08-14 11:18:54 -07:00
Mateusz Mikuła 31f71dedee Routine dependencies update (#533)
* Update dependencies

* Replace deprecated tempdir with tempfile
2018-08-10 12:37:45 -07:00
Stjepan Glavina 989262fe6e Enable sanitizer tests for tokio-threadpool (#537)
Closes #536.
2018-08-10 19:13:51 +02:00
Carl Lerche d91c775f36 Remove dead futures2 code. (#538)
The futures 0.2 crate is not intended for widespread usage. Also, the
futures team is exploring the compat shim route.

If futures 0.3 support is added to Tokio 0.1, then a different
integration route will be explored, making the current code unhelpful.
2018-08-09 21:56:53 -07:00
Stjepan Glavina 96b556fbff Steal multiple tasks from another worker at a time (#534)
* Steal multiple tasks from another worker at a time
* Better spinning and failing pop
* Update crossbeam-deque and simplify spinning
2018-08-09 12:14:13 -07:00
Stjepan Glavina fd36054ae4 Use a scalable RW lock in tokio-reactor (#517) 2018-08-09 11:23:45 -07:00
Carl Lerche 89d6bfc5cb Bump tokio-tls to v0.2.0 (#531)
This prepares the crate for release.
2018-08-08 10:17:28 -07:00
David Kellum decc83e959 Update to crossbeam-utils 0.5.0, fix imports (#519) 2018-08-08 08:57:25 -07:00
Sean McArthur afcfefd7e3 Move tokio-tls into workspace (#529) 2018-08-08 08:36:17 -07:00
Roman c89b0b4c8c Fix num CPUs in threadpool::builder::Builder::new (#530)
Closes #400
2018-08-08 08:33:01 -07:00
Stjepan Glavina 6b1e4ab0a3 Implement Error for a few error types (#511) 2018-08-07 19:45:58 -07:00
David Kellum 4153cc4076 Fix more rustdoc links (#518)
* Fix a cut-paste error with -reactor rustdoc links
* Fix more ::reactor rustdoc broken links
* Minor rustdoc typo
* Consistently reference std::io::{Read, Write} in rustdoc/links
2018-08-07 19:45:39 -07:00
Serho Liu 5304557d1d Fix tokio threadpool readme examples (#521) 2018-08-07 19:44:45 -07:00
Andrew Cann fdb2f61357 Udp socket readiness methods (#522) 2018-08-07 19:41:27 -07:00
Carl Lerche e964c4136c Bump subcrate versions (#524)
* tokio-current-thread 0.1.1
* tokio-executor 0.1.3
* tokio-fs 0.1.3
* tokio-reactor 0.1.3
* tokio-tcp 0.1.1
* tokio-timer 0.2.5
2018-08-06 20:36:50 -07:00
Brian Olsen 0490280d66 tokio-fs: Add async versions of most of std::fs (#494)
* create_dir
* create_dir_all
* hard_link
* read_dir
* read_link
* remove_dir
* remove_file
* rename
* set_permissions that works with path
* symlink_metadata
* symlink on unix
* symlink_dir on windows
* symlink_file on windows
2018-07-31 21:39:27 -07:00
Sam Rijs 0f76470172 detect and handle recursive calls to DefaultExecutor (#473) 2018-07-30 20:59:08 -07:00
Stjepan Glavina 9352249c3e Terminate backup threads when idle (#489) 2018-07-30 20:48:53 -07:00
Stjepan Glavina e5b2681513 Fix a race in thread wakeup (#507) 2018-07-30 20:46:46 -07:00
Stjepan Glavina 629c9f0698 Small fixes (#508)
* Make Shutdown public
* Remove unused import
* Fix documentation mistake
* Fix typo
2018-07-30 20:46:04 -07:00
Alan Somers 5d0d2a2e12 Ignore tokio-uds's test_socket_pair on FreeBSD. (#493)
It requires FreeBSD 12.0 or later.  Also, fix a spelling mistake in a
comment.
2018-07-24 14:00:01 -07:00
kohensu ad4693a18f Fix the doc of read_to_end method (#482) 2018-07-24 13:57:15 -07:00
Laurențiu Nicola c85bde3170 tokio: expose tokio_fs::metadata (#479) 2018-07-24 13:56:42 -07:00
Jon Gjengset 1e90e27720 Count in-transit spawned futures to current thread executor as pending (#478) 2018-07-24 13:49:01 -07:00
Carl Lerche f212a2ab9d Fix Weak tsan whitelist (#505) 2018-07-24 13:37:48 -07:00
Michal 'vorner' Vaner 84db325628 RunError and few more error types implements Error (#501)
This allows them to be used with things like `failure`.
2018-07-24 13:27:57 -07:00
Douman 365efec24a Add Interval::interval shortcut for a better usability (#492) 2018-07-23 23:08:49 -07:00
David Kellum 491f15827b General rustdoc improvements (#450)
* Normalize links to docs.rs/CRATE/M.N/...

docs.rs is smart enough to show docs for the latest M.N.P release when
M.N is used in the link. For example:

  https://docs.rs/mio/0.6/mio/struct.Poll.html

..will show mio 0.6.14 and later docs. While using the `M.N.*`
(ASTERISK) syntax also works, `M.N` is the more common usage, so
standarize a few existing links to that format.

* Fix missing or malformed rustdoc links

* executor lib rustdoc minor format change

* Promote tokio-threadpool crate level comments to rustdoc

* Replace hidden tokio::executor::thread_pool docs with deprecation note

* Fix typo/simplify util module rustdoc

* Reuse some tokio::executor::thread_pool rustdoc for the crate

Relates to #421
2018-07-22 13:35:30 -07:00
Stjepan Glavina c17ecb53e7 Pad fields to cacheline size to avoid false sharing (#475) 2018-07-16 14:22:48 -07:00
Jon Gjengset 6ba8e7621d Add free block_on_all in current thread Runtime (#477) 2018-07-11 15:32:58 -07:00
Laurențiu Nicola 39c95d6206 tokio-fs: Bump version to 0.1.2 (#469)
* Add a couple of missing full stops in the documentation
2018-07-11 15:09:37 -07:00
Sam Rijs 78b6bd4ca5 implement Send and Sync for DefaultExecutor (#472)
Fxes #376
2018-07-11 12:35:32 -07:00
Richard Dodd (dodj) b3ff9e315c Update lib.rs (#471)
Fix build failure on nightly (combination of warning for "cannot be resolved" and lint deny(warnings))
2018-07-11 12:30:51 -07:00
Stjepan Glavina 990186ec9d Optimize spinning in Worker::run (#470) 2018-07-11 12:30:26 -07:00
Stjepan Glavina 19da6ff59a New version of crossbeam-deque (#468) 2018-07-11 12:24:10 -07:00
Roman 36c817f0c3 Update rand dep from 0.4 to 0.5 (#458) 2018-07-11 12:14:40 -07:00
David Kellum 35123f7ae4 Additional details for tokio-fs rustdoc (#454) 2018-07-11 12:13:16 -07:00
João Oliveira 54b7c1b10d tokio-tcp: add tokio::net::TcpStream::try_clone (#448) 2018-07-11 11:54:08 -07:00
Patrick Barrett e6fc3d209d return NotReady when recv_from wouldblock in uds (#452) 2018-07-11 11:37:57 -07:00
Carl Lerche f98b81e527 Bump minimum supported Rust to 1.25. (#465)
Currenty, 1.27 is the latest released Rust version.
2018-07-06 14:14:05 -07:00
Stjepan Glavina dc7202cfa9 Replace XorShiftRng with a custom RNG (#466) 2018-07-06 13:33:52 -07:00
Carl Lerche f1a7caea3f Bump tokio-threadpool to v0.1.5 (#462) 2018-07-05 10:22:03 -07:00
Stjepan Glavina b019532bc2 Implement status() for DefaultExecutor (#463) 2018-07-05 10:19:49 -07:00
Stjepan Glavina 7fb579c667 Fix a race in thread wakeup (#459) 2018-07-03 16:28:50 -07:00
Stjepan Glavina dbefa67058 Make WorkerId public (#460) 2018-07-02 13:34:08 -07:00
Roman 24d99c029e Add a verbose error message for BlockingError (#451)
Add a verbose error message for EnterError while trying to run
tokio_threadpool::blocking on a current_thread::Runtime
2018-06-26 08:37:48 -07:00
Laurențiu Nicola 3fecd0154c Add an explicit wait for the test to finish (#445) 2018-06-22 14:07:02 -07:00
Laurențiu Nicola 0440343a11 tokio-fs: add changelog for 0.1.2 (#444) 2018-06-22 14:06:50 -07:00
Roman 3cf56b7bfa Fix unneeded mut and some deprecated api (#442) 2018-06-21 09:47:21 -07:00
Roman 7153d8d6ce Add a verbose error message for EnterError (#441)
Add a verbose error message for EnterError while trying to run an
executor while another executor is already running.

Fixes: #410
2018-06-21 09:46:45 -07:00
Laurențiu Nicola ecfe2f6a05 tokio-fs: add tokio_fs::File::seek (#434) 2018-06-21 09:43:35 -07:00
Laurențiu Nicola 5753553ba3 Move metadata to a submodule (#439) 2018-06-21 09:41:38 -07:00
Laurențiu Nicola 04a4bfd455 tokio-fs: add tokio_fs::metadata (#433) 2018-06-20 13:12:06 -07:00
Jake Goulding b2f77dcebe Add a dedicated Future for retrieving the metadata of a file (#385) 2018-06-18 16:00:43 -07:00
Carl Lerche 85cf47de86 Enable backtraces in CI & disable TSAN (#436)
This PR enables backtraces when running tests and disables tsan for the thread pool.

The thread sanitizer was generating too many false positives. Once #329 lands, then it can
be re-enabled.
2018-06-18 15:15:45 -07:00
Steven Fackler 45bcea6c4f Reexport tokio_uds::ConnectFuture (#430) 2018-06-18 13:26:06 -07:00
Carl Lerche 3fac7ce68c Add some thread pool docs (#421) 2018-06-15 15:20:25 -07:00
Marc-Antoine Perennou 71c8f561e3 runtime: add block_on_all (#398)
Signed-off-by: Marc-Antoine Perennou <[email protected]>
2018-06-14 22:13:39 -07:00
Sean McArthur 011ebf44eb Implement Executor for Box<E: Executor> (#420) 2018-06-14 16:28:23 -07:00
Carl Lerche c25ea78ec9 Bump version of a number of sub crates (#414)
This includes:

* tokio-codec (0.1.0)
* tokio-current-thread (0.1.0)
* tokio-fs (0.1.1)
* tokio-io (0.1.7)
* tokio-reactor (0.1.2)
* tokio-udp (0.1.1)
2018-06-13 10:24:56 -07:00
Carl Lerche 2e0cd292d2 Fix some broken doc links (#413) 2018-06-13 09:02:46 -07:00
Sylwek 4ebaf18c27 Typo (#415) 2018-06-13 09:02:34 -07:00
Carl Lerche ab07733d66 Deprecate executor re-exports (#412) 2018-06-12 14:41:12 -07:00
Mat Sadler d1f825ca13 Add OpenOptions to tokio-fs (#390)
Add an `OpenOptions` struct to `tokio-fs` that mirrors the one found in
`std`. Also provide a conversion from a `std` instance to a Tokio instance.
2018-06-12 10:47:24 -07:00
Laurențiu Nicola 4cf7d73b22 tokio-fs: add into_std (#403) 2018-06-12 10:40:43 -07:00
jpbriquet 2cd854c2c7 tokio-current-thread crate (#370)
Extract `tokio::executor::current_thread` to a tokio-current-thread
crate. Deprecated fns stay in the old location. The new crate only
contains thee most recent API.
2018-06-12 10:26:03 -07:00
Carl Lerche ba05c39d65 Fix a deadlock that can happen when shutting down (#409)
There is a deadlock that can occur when the concurrent runtime shuts
down. This patch adds a test and fix.

Fixes #401.
2018-06-12 09:41:18 -07:00
Alyssa Ross 64b8884911 Fix typo in comment (#402) 2018-06-11 15:26:55 -07:00
pravic d391e63418 Duplicated word in documentation. (#405) 2018-06-11 15:17:09 -07:00
Carl Lerche 8d8c895a1c Remove tokio-codec dependency from tokio (#397)
This will be added again later once types are re-exported.
2018-06-08 09:56:40 -07:00
Carl Lerche dba5c27296 Bump version to v0.1.7 (#396)
This also bumps the versions of:

* tokio-threadpool
* tokio-timer
2018-06-06 20:14:35 -07:00
Carl Lerche db620b42ec Another attempt at abstracting Instant::now (#381)
Currently, the timer uses a `Now` trait to abstract the source of time.
This allows time to be mocked out. However, the current implementation
has a number of limitations as represented by #288 and #296.

The main issues are that `Now` requires `&mut self` which prevents a
value from being easily used in a concurrent environment. Also, when
wanting to write code that is abstract over the source of time, generics
get out of hand.

This patch provides an alternate solution. A new type, `Clock` is
provided which defaults to `Instant::now` as the source of time, but
allows configuring the actual source using a new iteration of the `Now`
trait. This time, `Now` is `Send + Sync + 'static`. Internally, `Clock`
stores the now value in an `Arc<Now>` value, which introduces dynamism
and allows `Clock` values to be cloned and be `Sync`.

Also, the current clock can be set for the current execution context
using the `with_default` pattern.

Because using the `Instant::now` will be the most common case by far, it
is special cased in order to avoid the need to allocate an `Arc` and use
dynamic dispatch.
2018-06-06 16:04:39 -07:00
David Kellum 9013ed9bd4 Fix description of BlockingError as io::Error (#384) 2018-06-06 14:34:55 -07:00
Carl Lerche 06325fa63b Bump tokio-uds to v0.2.0 (#395) 2018-06-06 14:09:07 -07:00
Sebastian Dröge 0d41ba7a08 Implement a Send Handle for the single-threaded Runtime (#340)
Implement a Send'able Handle for the single-threaded `Runtime` and
`CurrentThread` executor to spawn new tasks from other threads.
2018-06-05 16:56:15 -07:00
Carl Lerche c07a7b26d3 Cleanup FramedParts in new tokio-codec (#394) 2018-06-05 15:31:01 -07:00
Bryan Burgers f723d10087 Create tokio-codec (#360)
Create a new tokio-codec crate with many of the contents of
`tokio_io::codec`.
2018-06-04 20:36:06 -07:00
Jon Gjengset 3d7263d3a0 Implement Runtime::block_on using oneshot (#391) 2018-06-04 20:09:17 -07:00
Carl Lerche 9caec1c15d Remove futures2 crate (#380) 2018-05-29 16:28:00 -07:00
Carl Lerche 703f07ca17 Remove threadpool disclaimer (#378) 2018-05-29 15:59:37 -07:00
Michal 'vorner' Vaner db9371126d Include a manually built runtime example (#306) 2018-05-29 14:44:28 -07:00
Carl Lerche eb1cf8fc9b Unpin Rust nightly version (#379) 2018-05-29 14:36:52 -07:00
Carl Lerche 4af6109398 Fix bug related to spawning optimization (#375)
The thread pool optimizes cases where a task currently running on the
pool spawns a new future. However, the optimization did not factor in
cases where two thread pools interacted.

This patch fixes the optimization and includes a test.

Fixes #342
2018-05-24 22:06:32 -07:00
Roman Zeyde 96f3ec903c Fix a small typo in README.md (#373) 2018-05-23 12:07:46 -07:00
Chris Pick 8c791fd0bf Fix Runtime::new's doc link to tokio::run (#371) 2018-05-22 15:29:15 -07:00
Rijenkii c0747a5fc1 tokio-io: Fix the link to the repository (#372) 2018-05-22 15:28:28 -07:00
Carl Lerche c8e710d39e Import tokio-uds (#365)
This imports tokio-uds from the dedicated repo.
2018-05-14 14:48:32 -07:00
Carl Lerche e281e4f4cb Remove fuchsia references as it is not supported. (#355) 2018-05-14 12:00:19 -07:00
Carl Lerche 6598334021 Add Gitter badge to README (#358) 2018-05-14 12:00:10 -07:00
main() 35f3351c97 Document Handle::default() behavior (#359) 2018-05-14 11:11:28 -07:00
Jason Davies 1f5bb121e2 Fix typo in doc comment. (#361) 2018-05-14 11:10:25 -07:00
sbstp 88801bb613 timer: add sleep free function (#347) 2018-05-11 09:16:08 -07:00
Carl Lerche a850063211 Handle::default() should lazily bind to reactor. (#350)
Currently, not specifying a `Handle` is different than using
`Handle::default()`. This is because `Handle::default()` will
immediately bind to the reactor for the current context vs. not
specifying a `Handle`, which binds to a reactor when it is polled.

This patch changes the `Handle::default()` behavior, bringing it inline
with actual defaults.

`Handle::current()` still immediately binds to the current reactor.

Fixes #307
2018-05-11 08:32:03 -07:00
Marek Kotewicz 14ec268b8a Fixed broken link in tokio-fs documentation (#352) 2018-05-11 08:31:06 -07:00
Thijs Vermeir 363b207f2b Fix typo in documentation (#346) 2018-05-08 11:44:50 -07:00
Julian Tescher 06b2c40222 Fix typos (#348) 2018-05-08 11:44:17 -07:00
Thijs Vermeir 68b82f5721 Fix typo in documentation (#341) 2018-05-04 07:06:47 -07:00
Thijs Vermeir 7cca6499a9 Fix typo in documentation (#338) 2018-05-03 10:28:48 -07:00
Carl Lerche 8235eefbf0 Fix some dependency versions (#337) 2018-05-02 13:12:33 -07:00
Carl Lerche 14b31bdba5 Bump version to v0.1.6 (#336) 2018-05-02 12:14:44 -07:00
Carl Lerche f768163982 Filesystem manipulation APIs. (#323)
This patch adds a new crate: tokio-fs. This crate provides a wrapper
around `std` functionality that can only be performed using blocking
operations. This primarily includes filesystem operations, but it also
includes standard input, output, and error access as these streams
cannot be safely switched to non-blocking mode in a portable way.

These wrappers call the `std` functions from within a `blocking`
annotation which allows the runtime to compensate for the fact that the
thread will potentially remain blocked in a system call.
2018-05-02 11:19:58 -07:00
Carl Lerche 7a2b5db15c Remove futures2 feature from Cargo.toml files (#334)
Currently, the state of the futures2 integration is pretty broken. This
patch removes the feature flag, preventing users from trying to use it.
In the future, it can be brought back when the implementation is fixed.
2018-05-02 10:48:58 -07:00
Roman 2465483845 Current thread runtime (#308)
This patch introduces a version of `Runtime` that runs all components on
the current thread. This allows users to spawn futures that do not implement
`Send`.
2018-05-02 09:40:42 -07:00
Stefan Bühler 6a0ecef81a Timer: always park nested Park (#327)
The nested `Park` might need to do some work, even if the duration is 0
seconds (e.g. a `Reactor`).

Similar to what #313 did for CurrentThread.
2018-05-01 16:33:41 -07:00
Stefan Bühler 6defeeb2ba current_thread: make underlying Park instance accessible 2018-05-01 14:33:38 -07:00
Stefan Bühler b36a73059d tokio-io: require bytes-0.4.7 for Buf::get_uint_be 2018-05-01 14:33:38 -07:00
Roman d1d4fe4d07 Stop using deprecated bytes APIs in tests (#324) (#331) 2018-04-30 10:02:48 -07:00
Carl Lerche 9aaa8f06d1 Stop using deprecated bytes APIs (#324)
This also adds a filter for another treiber stack expected data race. The
race is expected as part of the algorithm.
2018-04-28 12:25:22 -07:00
Sebastian Dröge 6ea00162b9 Make CurrentThread::turn() more fair by always parking with 0 timeout… (#313)
This ensures that all fd-based futures are put into the queue for the
current tick, if the CurrentThread is parking via the Reactor.

Otherwise, if there are queued up futures already, only those would be
polled in the turn. These futures could then notify others/themselves to
have the queue still non-empty on the next turn. Which then potentially
allows the reactor to never be polled, and thus fd-based futures are
never queued up and polled.

Also return in the Turn return value whether any futures were polled at
all, which allows the caller to know if any work was done at all in this
turn and based on that adjust behavior.
2018-04-25 10:37:18 -07:00
Carl Lerche 61d635e8ad Threadpool blocking (#317)
This patch adds a `blocking` to `tokio-threadpool`. This function serves
as a way to annotate sections of code that will perform blocking
operations. This informs the thread pool that an additional thread needs
to be spawned to replace the current thread, which will no longer be
able to process the work queue.
2018-04-15 12:29:22 -07:00
Carl Lerche 372400ed34 Add additional timer::Error docs. (#311)
Closes #302
2018-04-10 14:28:37 -07:00
Roman ba9d849ef0 Fix warning: variable does not need to be mutable (#309) 2018-04-10 13:33:05 -07:00
Roman 5b677934fe Add example that prints each packet from tcp client (#301) 2018-04-10 13:08:55 -07:00
Sam Rijs dbcd8353b0 Update futures2 to use the futures 0.2 release (#304) 2018-04-08 20:23:33 -07:00
Carl Lerche 3be6b69e1b Refactor threadpool task types (#300)
Replaces homegrown Arc with std Arc

Is this safer? Unknown. At least we don't have to maintain an arc
implementation anymore. This will also make it easier to filter out tsan
false positives.

Also split task/mod.rs into multiple files.
2018-04-05 10:57:05 -07:00
Carl Lerche 0bcf9b0ae6 ThreadPool refactoring (#299) 2018-04-04 13:30:54 -07:00
Carl Lerche c715739599 Add arc::Weak to tsan filter. (#298) 2018-04-04 12:54:49 -07:00
David 6aea9c43e8 Update Cargo.toml (#293) 2018-04-04 09:18:56 -07:00
Igor Gnatenko 82f6a52d1a threadpool: bump minimal version of executor (#292) 2018-04-04 09:18:40 -07:00
Leandro Pacheco a6b307cfbe re-export io::{ReadHalf/WriteHalf} timer::Error (#290) 2018-04-04 09:18:12 -07:00
Roman dcb20b289c Build 32/64-bit Linux and FreeBSD on Travis CI (#286) 2018-04-04 08:37:28 -07:00
Carl Lerche 79afc7ee68 Threadpool refactor (#294)
* Switch worker lifecycle to an enum
* Move some files around
* Rename State -> PoolState
2018-04-03 22:35:59 -07:00
Kam Y. Tse 3ba5595233 Fix typo (#275) 2018-04-02 13:11:06 -07:00
Carl Lerche 7232ba6d55 Bump tokio-timer to v0.2.1 (#287) 2018-04-02 11:06:22 -07:00
Roman a14de909eb Build both x86 and x64 on Windows (#282) 2018-04-02 09:37:56 -07:00
laizy d8789cd379 fix panic in chat example (#279) 2018-04-02 09:00:40 -07:00
Roman 8d4be0361e Fix unused variable in tokio-threadpool\tests\threadpool.rs:581:9 (#284) 2018-04-02 09:00:14 -07:00
Daniel Griffen 3f2710397d Fix tokio-timer on 32bit systems (#274) 2018-04-02 08:53:43 -07:00
Roman 8895a7d3ab Fix Appveyor badge on crates.io page (#280) 2018-04-01 16:16:22 -07:00
Carl Lerche 10cb9dd468 Actually bump tokio to v0.1.5 (#273) 2018-03-30 15:37:52 -07:00
Carl Lerche 2ca214bd2c Fix tokio dependency versions (#272) 2018-03-30 15:32:25 -07:00
Carl Lerche 2a01c26d58 Bump version to v0.1.5 (#271)
This also bumps:

* tokio-executor to v0.1.2
* tokio-threadpool to v0.1.2
* tokio-timer to v0.2.0
2018-03-30 15:28:44 -07:00
Carl Lerche ea172537aa Rename Sleep to Delay (#270)
This patch renames `Sleep` from tokio-timer and the tokio facade to
`Delay`. Given that the future does not actually put anything to sleep,
the `Delay` name feels more appropriate.

Fixes #263
2018-03-30 14:21:48 -07:00
Carl Lerche baa2502ec6 Integrate timers with runtime. (#266)
This patch integrate the new timer implementation with the runtime by
initializing a timer per worker thread. This allows minimizing the
amount of synchronization needed for using timers.
2018-03-30 11:50:02 -07:00
Carl Lerche d4d17392fe Remove println from tests (#267) 2018-03-29 20:58:02 -07:00
Carl Lerche 6807363efd Fix permissions (#268) 2018-03-29 20:42:59 -07:00
Carl Lerche 1c5d131245 Allow customizing the threadpool's parker (#264)
* Allow customizing the threadpool's parker

This patch allows the user of threadpool to customize how the worker
threads park themselves. This allows custom parking logic to be
injected. For example, this allows embedding a timer on each worker
thread.

* Call `park` instance every so often.

Since the `park` is now customizable, it might have logic that must be
called every so often. For example, a timer might have timeouts that it
must expire.

Currently, if a worker is very busy, it won't call into the `park`
instance. This patch changes this so that after every 32 task
invocations, `park` is called with a duration of zero.
2018-03-29 13:47:08 -07:00
Carl Lerche 19500f7df8 Provide a timer implementation (#249)
This patch adds a new crate: tokio-timer. This crate provides an
efficient timer implemeentation designed for use in Tokio based
applications.

The timer users a hierarchical hashed timer wheel algorithm with six
levels, each having 64 slots. This allows the timer to have a resolution
of 1ms while maintaining O(1) complexity for insert, removal, and firing
of timeouts.

There already exists a tokio-timer crate. This is a complete rewrite
which solves the outstanding problems with the existing tokio-timer
library.

Closes #146.
2018-03-28 22:26:47 -07:00
Roman ad189826f4 Split tokio-threadpool lib.rs into files (#233)
* Builder -> src/builder.rs
* Callback -> src/callback.rs
* Config -> src/config.rs
* Futures2Wake -> src/futures2_wake.rs
* Inner -> src/inner.rs
* Notifier-> src/notifier.rs
* Sender -> src/sender.rs
* Shutdown -> src/shutdown.rs
* ShutdownTask -> src/shutdown_task.rs
* SleepStack -> src/sleep_stack.rs
* State -> src/state.rs
* ThreadPool -> src/thread_pool.rs
* Worker -> src/worker.rs
* WorkerEntry -> src/worker_entry.rs
* WorkerState -> src/worker_state.rs
2018-03-27 15:56:21 -07:00
Klaus Purer a612736f54 fix(cargo): Bump dependencies so that Tokio compiles with minimal versions (#258) 2018-03-27 15:47:11 -07:00
Carl Lerche bda8dd5113 Update futures2 Cargo.toml (#256) 2018-03-27 15:46:19 -07:00
Sam Rijs 415a786049 Fix unstable-futures feature flag propagation (#261) 2018-03-27 15:46:02 -07:00
Carl Lerche 2edc35a45d Disable future 0.2 tests (#259) 2018-03-24 14:01:41 -07:00
Carl Lerche 9cffda59c9 Bump version to v0.1.4 (#252)
This also bumps:

* tokio-executor: v0.1.1
* tokio-reactor: v0.1.1
* tokio-threadpool: v0.1.1
2018-03-23 10:34:42 -07:00
Carl Lerche 23d95b44e7 Change hammer test consts (#255) 2018-03-23 10:27:58 -07:00
Denis 69e45f4be4 Fix a typo in tokio_reactor::PollEvented (#250)
`read_readiness` -> `write_readiness` in *write* methods
2018-03-22 20:15:31 -07:00
Denis 6bdfa159a7 Fix connect example for UDP (#241)
Close #241
2018-03-22 10:02:01 -07:00
Michal 'vorner' Vaner 5d87a9cee1 Docs: warn about errors from TcpListener::incoming (#247) 2018-03-22 10:01:05 -07:00
Denis 16d3540ce9 Add UDP client example (send/recv_dgram) (#239) 2018-03-22 10:00:27 -07:00
Sean McArthur e5ebd02885 implement poll_vectored_* and initializer method for futures2 (#242) 2018-03-22 09:59:13 -07:00
Carl Lerche 08c21e7bac Fix race condition related bugs (#243)
* Fix races.

This mostly pulls in changes from rust-lang-nursery/futures-rs#881, but
also updates Registration to be a bit more obvious as to what is going
on.

* Reduce spurious wakeups caused by Reactor

This patch adds an ABA guard on token values before registering them
with Mio. This allows catching token reuse and avoid the notification.

This is needed for OS X as the notification is used to determine that a
TCP connect has completed. A spurious notification can potentially cause
write failures.
2018-03-22 09:57:40 -07:00
Aaron Turon 8786741ba9 Update to futures 0.2.0-beta (#246) 2018-03-21 14:21:02 -07:00
Roman 494f0dc176 Runtime builder (#234)
* Split runtime module into files
* Add runtime::Builder to set up thread pool.
2018-03-21 11:23:36 -07:00
Cyril Plisko df9025594c Lapsus clavis (#245) 2018-03-21 11:17:16 -07:00
Aaron Turon 7b1306e6c2 Add top-level tests for futures 0.2 integration (#231) 2018-03-15 16:38:12 -07:00
Sean McArthur b7f4e337be set Runtime thread pool name prefix (#232) 2018-03-15 16:37:32 -07:00
Hiroaki Nakamura d1046db735 Fix condition for updating the current date buffer (#230) 2018-03-15 09:29:55 -07:00
Sam Rijs 923a80e098 Move tokio::net module into tokio tcp/udp crates (#224) 2018-03-14 09:38:59 -07:00
Carl Lerche 64435f5b35 Travis: Move before_deploy out of matrix block (#229) 2018-03-13 15:58:53 -07:00
Aaron Turon d304791c0e Simultaneous futures compat (#172)
This patch adds opt-in support for futures 0.2.
2018-03-13 13:57:35 -07:00
Carl Lerche 5846b3fc2a Reduce AtomicTask spurious notifications (#227) 2018-03-13 13:28:41 -07:00
Carl Lerche 8eb3e58b7d Shutdown the runtime on drop (#214)
Currently, the runtime does not shutdown if the runtime handle is
dropped. This can happen during a panic or when the value is simply
dropped.

This patch forces the runtime to shutdown if it is not explicitly
shutdown.

Fixes #209
2018-03-13 13:14:51 -07:00
Carl Lerche c0a2cc1f9e Add LICENSE file to all sub crates (#226)
* Add LICENSE file to all sub crates
* Update links in README
2018-03-13 13:14:28 -07:00
Carl Lerche 61b2889881 Try compiling Tokio on FreeBSD (#228) 2018-03-13 13:14:12 -07:00
hcpl 5dab821b29 Fix docs markup (#225) 2018-03-13 11:43:05 -07:00
Wesley Moore 2abeff01a5 Fix build on FreeBSD (#218) 2018-03-13 11:24:17 -07:00
Carl Lerche 96a542451d Handle futures that panic on a threadpool (#216)
If a future panics from within the context of a thread pool, the pool
should not be impacted. To do this, polling the future is wrapped with a
catch_unwind. Extra care is taken to ensure that `thread::panicking()`
is set from within the future's drop handle.

Fixes #209
2018-03-13 09:44:14 -07:00
Gray Olson 95899e007d Update comment in udp-codec example (#222) 2018-03-12 10:39:32 -07:00
Jeehoon Kang e6e3c49e0e Remove uses of futures_cpupool (#220) 2018-03-11 11:37:10 -07:00
Carl Lerche 4d514b7eb3 Relicense Tokio exclusively under the MIT license. (#215)
This patch relicenses the Tokio project exclusively under the MIT
license. Before this, the project was dual licensed under MIT and Apache
2. As such, switching to only MIT is permitted.

Fixes #202
2018-03-09 20:07:09 -08:00
Tosil Velkov 2a1585157e Fix wrong file link in examples readme.md (#208) 2018-03-09 12:26:09 -08:00
Igor Gnatenko 189d6baac4 tokio-threadpool: bump rand to 0.4 (#205) 2018-03-09 12:25:38 -08:00
Igor Gnatenko 3ad27e99ec tokio-reactor: bump mio to 0.6.14 (#204)
With 0.6.13 it doesn't compile:
no method named `as_usize` found for type `mio::Ready` in the current scope
2018-03-09 12:25:16 -08:00
479 changed files with 51352 additions and 16017 deletions
-16
View File
@@ -1,16 +0,0 @@
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host x86_64-pc-windows-msvc
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- if NOT "%TARGET%" == "x86_64-pc-windows-msvc" rustup target add %TARGET%
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --target %TARGET%
+43
View File
@@ -0,0 +1,43 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_PREEMPTIONS: 2
RUSTFLAGS: -Dwarnings
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
# TODO: switch back to nightly
- sh rustup.sh -y --default-toolchain nightly-2019-08-21
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
test_script:
- . $HOME/.cargo/env
- cargo test --all
- cargo doc --all --no-deps
# TODO: Re-enable
# i686_test_script:
# - . $HOME/.cargo/env
# - |
# cargo test --all --exclude tokio-tls --exclude tokio-macros --target i686-unknown-freebsd
+51
View File
@@ -0,0 +1,51 @@
<!--
Thank you for reporting an issue.
Please fill in as much of the template below as you're able.
-->
## Version
<!--
List the versions of all `tokio` crates you are using. The easiest way to get
this information is using `cargo-tree`.
`cargo install cargo-tree`
(see install here: https://github.com/sfackler/cargo-tree)
Then:
`cargo tree | grep tokio`
-->
## Platform
<!---
Output of `uname -a` (UNIX), or version and 32 or 64-bit (Windows)
-->
## Subcrates
<!--
If known, please specify the affected Tokio sub crates. Otherwise, delete this
section.
-->
## Description
<!--
Enter your issue details below this comment.
One way to structure the description:
<short summary of the bug>
I tried this code:
<code sample that causes the bug>
I expected to see this happen: <explanation>
Instead, this happened: <explanation>
-->
+23
View File
@@ -0,0 +1,23 @@
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.
Bug fixes and new features should include tests.
Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->
## Motivation
<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->
## Solution
<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
-40
View File
@@ -1,40 +0,0 @@
---
language: rust
sudo: false
matrix:
include:
- rust: 1.21.0
- rust: stable
before_deploy: cargo doc --all --no-deps
- os: osx
- rust: beta
- rust: nightly
script:
- |
if [[ "$TRAVIS_RUST_VERSION" == nightly ]]
then
cargo build --benches --all
fi
- cargo test --all
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = linux
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
-21
View File
@@ -1,21 +0,0 @@
# 0.1.3 (March 09, 2018)
* Fix `CurrentThread::turn` to block on idle (#212).
# 0.1.2 (March 09, 2018)
* Introduce Tokio Runtime (#141)
* Provide `CurrentThread` for more flexible usage of current thread executor (#141).
* Add Lio for platforms that support it (#142).
* I/O resources now lazily bind to the reactor (#160).
* Extract Reactor to dedicated crate (#169)
* Add facade to sub crates and add prelude (#166).
* Switch TCP/UDP fns to poll_ -> Poll<...> style (#175)
# 0.1.1 (February 09, 2018)
* Doc fixes
# 0.1.0 (February 07, 2018)
* Initial crate released based on [RFC](https://github.com/tokio-rs/tokio-rfcs/pull/3).
+443
View File
@@ -0,0 +1,443 @@
# Contributing to Tokio
:balloon: Thanks for your help improving the project! We are so happy to have
you!
There are opportunities to contribute to Tokio at any level. It doesn't matter if
you are just getting started with Rust or are the most weathered expert, we can
use your help.
**No contribution is too small and all contributions are valued.**
This guide will help you get started. **Do not let this guide intimidate you**.
It should be considered a map to help you navigate the process.
You may also get help with contributing in the [dev channel][dev], please join
us!
[dev]: https://gitter.im/tokio-rs/dev
## Conduct
The Tokio project adheres to the [Rust Code of Conduct][coc]. This describes
the _minimum_ behavior expected from all contributors.
[coc]: https://github.com/rust-lang/rust/blob/master/CODE_OF_CONDUCT.md
## Contributing in Issues
For any issue, there are fundamentally three ways an individual can contribute:
1. By opening the issue for discussion: For instance, if you believe that you
have uncovered a bug in Tokio, creating a new issue in the tokio-rs/tokio
issue tracker is the way to report it.
2. By helping to triage the issue: This can be done by providing
supporting details (a test case that demonstrates a bug), providing
suggestions on how to address the issue, or ensuring that the issue is tagged
correctly.
3. By helping to resolve the issue: Typically this is done either in the form of
demonstrating that the issue reported is not a problem after all, or more
often, by opening a Pull Request that changes some bit of something in
Tokio in a concrete and reviewable manner.
**Anybody can participate in any stage of contribution**. We urge you to
participate in the discussion around bugs and participate in reviewing PRs.
### Asking for General Help
If you have reviewed existing documentation and still have questions or are
having problems, you can open an issue asking for help.
In exchange for receiving help, we ask that you contribute back a documentation
PR that helps others avoid the problems that you encountered.
### Submitting a Bug Report
When opening a new issue in the Tokio issue tracker, users will be presented
with a [basic template][template] that should be filled in. If you believe that you have
uncovered a bug, please fill out this form, following the template to the best
of your ability. Do not worry if you cannot answer every detail, just fill in
what you can.
The two most important pieces of information we need in order to properly
evaluate the report is a description of the behavior you are seeing and a simple
test case we can use to recreate the problem on our own. If we cannot recreate
the issue, it becomes impossible for us to fix.
In order to rule out the possibility of bugs introduced by userland code, test
cases should be limited, as much as possible, to using only Tokio APIs.
See [How to create a Minimal, Complete, and Verifiable example][mcve].
[mcve]: https://stackoverflow.com/help/mcve
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Triaging a Bug Report
Once an issue has been opened, it is not uncommon for there to be discussion
around it. Some contributors may have differing opinions about the issue,
including whether the behavior being seen is a bug or a feature. This discussion
is part of the process and should be kept focused, helpful, and professional.
Short, clipped responses—that provide neither additional context nor supporting
detail—are not helpful or professional. To many, such responses are simply
annoying and unfriendly.
Contributors are encouraged to help one another make forward progress as much as
possible, empowering one another to solve issues collaboratively. If you choose
to comment on an issue that you feel either is not a problem that needs to be
fixed, or if you encounter information in an issue that you feel is incorrect,
explain why you feel that way with additional supporting context, and be willing
to be convinced that you may be wrong. By doing so, we can often reach the
correct outcome much faster.
### Resolving a Bug Report
In the majority of cases, issues are resolved by opening a Pull Request. The
process for opening and reviewing a Pull Request is similar to that of opening
and triaging issues, but carries with it a necessary review and approval
workflow that ensures that the proposed changes meet the minimal quality and
functional guidelines of the Tokio project.
## Pull Requests
Pull Requests are the way concrete changes are made to the code, documentation,
and dependencies in the Tokio repository.
Even tiny pull requests (e.g., one character pull request fixing a typo in API
documentation) are greatly appreciated. Before making a large change, it is
usually a good idea to first open an issue describing the change to solicit
feedback and guidance. This will increase the likelihood of the PR getting
merged.
### Tests
If the change being proposed alters code (as opposed to only documentation for
example), it is either adding new functionality to Tokio or it is fixing
existing, broken functionality. In both of these cases, the pull request should
include one or more tests to ensure that Tokio does not regress in the future.
There are two ways to write tests: integration tests and documentation tests
(Tokio avoids unit tests as much as possible).
#### Integration tests
Integration tests go in the same crate as the code they are testing. Each sub
crate should have a `dev-dependency` on `tokio` itself. This makes all Tokio
utilities available to use in tests, no matter the crate being tested.
The best strategy for writing a new integration test is to look at existing
integration tests in the crate and follow the style.
#### Documentation tests
Ideally, every API has at least one [documentation test] that demonstrates how to
use the API. Documentation tests are run with `cargo test --doc`. This ensures
that the example is correct and provides additional test coverage.
The trick to documentation tests is striking a balance between being succinct
for a reader to understand and actually testing the API.
Same as with integration tests, when writing a documentation test, the full
`tokio` crate is available. This is especially useful for getting access to the
runtime to run the example.
The documentation tests will be visible from both the crate specific
documentation **and** the `tokio` facade documentation via the re-export. The
example should be written from the point of view of a user that is using the
`tokio` crate. As such, the example should use the API via the facade and not by
directly referencing the crate.
The type level example for `tokio_timer::Timeout` provides a good example of a
documentation test:
```
/// // import the `timeout` function, usually this is done
/// // with `use tokio::prelude::*`
/// use tokio::prelude::FutureExt;
/// use futures::Stream;
/// use futures::sync::mpsc;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = mpsc::unbounded();
/// # tx.unbounded_send(()).unwrap();
/// # drop(tx);
///
/// let process = rx.for_each(|item| {
/// // do something with `item`
/// # drop(item);
/// # Ok(())
/// });
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// process.timeout(Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
Given that this is a *type* level documentation test and the primary way users
of `tokio` will create an instance of `Timeout` is by using
`FutureExt::timeout`, this is how the documentation test is structured.
Lines that start with `/// #` are removed when the documentation is generated.
They are only there to get the test to run. The `block_on_all` function is the
easiest way to execute a future from a test.
If this were a documentation test for the `Timeout::new` function, then the
example would explicitly use `Timeout::new`. For example:
```
/// use tokio::timer::Timeout;
/// use futures::Future;
/// use futures::sync::oneshot;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = oneshot::channel();
/// # tx.send(()).unwrap();
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// Timeout::new(rx, Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
### Commits
It is a recommended best practice to keep your changes as logically grouped as
possible within individual commits. There is no limit to the number of commits
any single Pull Request may have, and many contributors find it easier to review
changes that are split across multiple commits.
That said, if you have a number of commits that are "checkpoints" and don't
represent a single logical change, please squash those together.
Note that multiple commits often get squashed when they are landed (see the
notes about [commit squashing]).
#### Commit message guidelines
A good commit message should describe what changed and why.
1. The first line should:
* contain a short description of the change (preferably 50 characters or less,
and no more than 72 characters)
* be entirely in lowercase with the exception of proper nouns, acronyms, and
the words that refer to code, like function/variable names
* be prefixed with the name of the sub crate being changed (without the `tokio-`
prefix) and start with an imperative verb. If modifying `tokio` proper,
omit the crate prefix.
Examples:
* timer: introduce `Timeout` and deprecate `Deadline`
* export `Encoder`, `Decoder`, `Framed*` from tokio_codec
2. Keep the second line blank.
3. Wrap all other lines at 72 columns (except for long URLs).
4. If your patch fixes an open issue, you can add a reference to it at the end
of the log. Use the `Fixes: #` prefix and the issue number. For other
references use `Refs: #`. `Refs` may include multiple issues, separated by a
comma.
Examples:
- `Fixes: #1337`
- `Refs: #1234`
Sample complete commit message:
```txt
subcrate: explain the commit in one line
Body of commit message is a few lines of text, explaining things
in more detail, possibly giving some background about the issue
being fixed, etc.
The body of the commit message can be several paragraphs, and
please do proper word-wrap and keep columns shorter than about
72 characters or so. That way, `git log` will show things
nicely even when it is indented.
Fixes: #1337
Refs: #453, #154
```
### Opening the Pull Request
From within GitHub, opening a new Pull Request will present you with a
[template] that should be filled out. Please try to do your best at filling out
the details, but feel free to skip parts if you're not sure what to put.
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Discuss and update
You will probably get feedback or requests for changes to your Pull Request.
This is a big part of the submission process so don't be discouraged! Some
contributors may sign off on the Pull Request right away, others may have
more detailed comments or feedback. This is a necessary part of the process
in order to evaluate whether the changes are correct and necessary.
**Any community member can review a PR and you might get conflicting feedback**.
Keep an eye out for comments from code owners to provide guidance on conflicting
feedback.
**Once the PR is open, do not rebase the commits**. See [Commit Squashing] for
more details.
### Commit Squashing
In most cases, **do not squash commits that you add to your Pull Request during
the review process**. When the commits in your Pull Request land, they may be
squashed into one commit per logical change. Metadata will be added to the
commit message (including links to the Pull Request, links to relevant issues,
and the names of the reviewers). The commit history of your Pull Request,
however, will stay intact on the Pull Request page.
## Reviewing Pull Requests
**Any Tokio community member is welcome to review any pull request**.
All Tokio contributors who choose to review and provide feedback on Pull
Requests have a responsibility to both the project and the individual making the
contribution. Reviews and feedback must be helpful, insightful, and geared
towards improving the contribution as opposed to simply blocking it. If there
are reasons why you feel the PR should not land, explain what those are. Do not
expect to be able to block a Pull Request from advancing simply because you say
"No" without giving an explanation. Be open to having your mind changed. Be open
to working with the contributor to make the Pull Request better.
Reviews that are dismissive or disrespectful of the contributor or any other
reviewers are strictly counter to the Code of Conduct.
When reviewing a Pull Request, the primary goals are for the codebase to improve
and for the person submitting the request to succeed. **Even if a Pull Request
does not land, the submitters should come away from the experience feeling like
their effort was not wasted or unappreciated**. Every Pull Request from a new
contributor is an opportunity to grow the community.
### Review a bit at a time.
Do not overwhelm new contributors.
It is tempting to micro-optimize and make everything about relative performance,
perfect grammar, or exact style matches. Do not succumb to that temptation.
Focus first on the most significant aspects of the change:
1. Does this change make sense for Tokio?
2. Does this change make Tokio better, even if only incrementally?
3. Are there clear bugs or larger scale issues that need attending to?
4. Is the commit message readable and correct? If it contains a breaking change
is it clear enough?
Note that only **incremental** improvement is needed to land a PR. This means
that the PR does not need to be perfect, only better than the status quo. Follow
up PRs may be opened to continue iterating.
When changes are necessary, *request* them, do not *demand* them, and **do not
assume that the submitter already knows how to add a test or run a benchmark**.
Specific performance optimization techniques, coding styles and conventions
change over time. The first impression you give to a new contributor never does.
Nits (requests for small changes that are not essential) are fine, but try to
avoid stalling the Pull Request. Most nits can typically be fixed by the Tokio
Collaborator landing the Pull Request but they can also be an opportunity for
the contributor to learn a bit more about the project.
It is always good to clearly indicate nits when you comment: e.g.
`Nit: change foo() to bar(). But this is not blocking.`
If your comments were addressed but were not folded automatically after new
commits or if they proved to be mistaken, please, [hide them][hiding-a-comment]
with the appropriate reason to keep the conversation flow concise and relevant.
### Be aware of the person behind the code
Be aware that *how* you communicate requests and reviews in your feedback can
have a significant impact on the success of the Pull Request. Yes, we may land
a particular change that makes Tokio better, but the individual might just not
want to have anything to do with Tokio ever again. The goal is not just having
good code.
### Abandoned or Stalled Pull Requests
If a Pull Request appears to be abandoned or stalled, it is polite to first
check with the contributor to see if they intend to continue the work before
checking if they would mind if you took it over (especially if it just has nits
left). When doing so, it is courteous to give the original contributor credit
for the work they started (either by preserving their name and email address in
the commit log, or by using an `Author: ` meta-data tag in the commit.
_Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
## Releasing
Since the Tokio project consists of a number of crates, many of which depend on
each other, releasing new versions to crates.io can involve some complexities.
When releasing a new version of a crate, follow these steps:
1. **Ensure that the release crate has no path dependencies.** When the HEAD
version of a Tokio crate requires unreleased changes in another Tokio crate,
the crates.io dependency on the second crate will be replaced with a path
dependency. Crates with path dependencies cannot be published, so before
publishing the dependent crate, any path dependencies must also be published.
This should be done through a form of depth-first tree traversal:
1. Starting with the first path dependency in the crate to be released,
inspect the `Cargo.toml` for the dependency. If the dependency has any
path dependencies of its own, repeat this step with the first such
dependency.
2. Begin the release process for the path dependency.
3. Once the path dependency has been published to crates.io, update the
dependent crate to depend on the crates.io version.
4. When all path dependencies have been published, the dependent crate may
be published.
To verify that a crate is ready to publish, run:
```bash
bin/publish --dry-run <CRATE NAME> <CRATE VERSION>
```
2. **Update Cargo metadata.** After releasing any path dependencies, update the
`version` field in `Cargo.toml` to the new version, and the `documentation`
field to the docs.rs URL of the new version.
3. **Update other documentation links.** Update the `#![doc(html_root_url)]`
attribute in the crate's `lib.rs` and the "Documentation" link in the crate's
`README.md` to point to the docs.rs URL of the new version.
4. **Update the changelog for the crate.** Each crate in the Tokio repository
has its own `CHANGELOG.md` in that crate's subdirectory. Any changes to that
crate since the last release should be added to the changelog. Change
descriptions may be taken from the Git history, but should be edited to
ensure a consistent format, based on [Keep A Changelog][keep-a-changelog].
Other entries in that crate's changelog may also be used for reference.
5. **Perform a final audit for breaking changes.** Compare the HEAD version of
crate with the Git tag for the most recent release version. If there are any
breaking API changes, determine if those changes can be made without breaking
existing APIs. If so, resolve those issues. Otherwise, if it is necessary to
make a breaking release, update the version numbers to reflect this.
6. **Open a pull request with your changes.** Once that pull request has been
approved by a maintainer and the pull request has been merged, continue to
the next step.
7. **Release the crate.** Run the following command:
```bash
bin/publish <NAME OF CRATE> <VERSION>
```
Your editor and prompt you to edit a message for the tag. Copy the changelog
entry for that release version into your editor and close the window.
[keep-a-changelog]: https://github.com/olivierlacan/keep-a-changelog/blob/master/CHANGELOG.md
+11 -56
View File
@@ -1,62 +1,17 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.3"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT/Apache-2.0"
readme = "README.md"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio/0.1"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio",
"tokio-buf",
"tokio-codec",
"tokio-executor",
"tokio-fs",
"tokio-io",
"tokio-reactor",
"tokio-threadpool",
"tokio-macros",
"tokio-net",
"tokio-sync",
"tokio-test",
"tokio-timer",
"tokio-tls",
"build-tests",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio" }
[dependencies]
tokio-io = { version = "0.1.6", path = "tokio-io" }
tokio-executor = { version = "0.1.0", path = "tokio-executor" }
tokio-reactor = { version = "0.1.0", path = "tokio-reactor" }
tokio-threadpool = { version = "0.1.0", path = "tokio-threadpool" }
bytes = "0.4"
log = "0.4"
mio = "0.6.14"
slab = "0.4"
iovec = "0.1"
futures = "0.1.18"
[dev-dependencies]
env_logger = { version = "0.4", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
tokio-io = { path = "tokio-io" }
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
-201
View File
@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+112 -65
View File
@@ -1,5 +1,7 @@
# Tokio
_NOTE_: Tokio's [`master`](https://github.com/tokio-rs/tokio) branch is currently in the process of moving to [`std::future::Future`](https://doc.rust-lang.org/std/future/trait.Future.html), for `v0.1.x` based tokio releases please check out the [`v0.1.x`](https://github.com/tokio-rs/tokio/tree/v0.1.x) branch.
A runtime for writing reliable, asynchronous, and slim applications with
the Rust programming language. It is:
@@ -14,25 +16,22 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[mit-url]: LICENSE
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[Guides](https://tokio.rs/docs/) |
[API Docs](https://docs.rs/tokio/0.2.0-alpha.2/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
## Overview
@@ -41,63 +40,83 @@ asynchronous applications with the Rust programming language. At a high
level, it provides a few major components:
* A multithreaded, work-stealing based task [scheduler].
* A [reactor] backed by the operating system's event queue (epoll, kqueue,
* A reactor backed by the operating system's event queue (epoll, kqueue,
IOCP, etc...).
* Asynchronous [TCP and UDP][net] sockets.
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/0.2.0-alpha.2/tokio/net/index.html
[scheduler]: https://docs.rs/tokio/0.2.0-alpha.2/tokio/runtime/index.html
## Example
A basic TCP echo server with Tokio:
```rust
extern crate tokio;
use tokio::prelude::*;
use tokio::io::copy;
use tokio::net::TcpListener;
use tokio::prelude::*;
fn main() {
// Bind the server's socket.
let addr = "127.0.0.1:12345".parse().unwrap();
let listener = TcpListener::bind(&addr)
.expect("unable to bind TCP listener");
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "127.0.0.1:8080".parse()?;
let mut listener = TcpListener::bind(&addr).unwrap();
// Pull out a stream of sockets for incoming connections
let server = listener.incoming()
.map_err(|e| eprintln!("accept failed = {:?}", e))
.for_each(|sock| {
// Split up the reading and writing parts of the
// socket.
let (reader, writer) = sock.split();
loop {
let (mut socket, _) = listener.accept().await?;
// A future that echos the data and returns how
// many bytes were copied...
let bytes_copied = copy(reader, writer);
tokio::spawn(async move {
let mut buf = [0; 1024];
// ... after which we'll print what happened.
let handle_conn = bytes_copied.map(|amt| {
println!("wrote {:?} bytes", amt)
}).map_err(|err| {
eprintln!("IO error {:?}", err)
});
// In a loop, read data from the socket and write the data back.
loop {
let n = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => return,
Ok(n) => n,
Err(e) => {
println!("failed to read from socket; err = {:?}", e);
return;
}
};
// Spawn the future as a concurrent task.
tokio::spawn(handle_conn)
// Write the data back
if let Err(e) = socket.write_all(&buf[0..n]).await {
println!("failed to write to socket; err = {:?}", e);
return;
}
}
});
// Start the Tokio runtime
tokio::run(server);
}
}
```
More examples can be found [here](examples).
More examples can be found [here](tokio/examples). Note that the `master` branch
is currently being updated to use `async` / `await`. The examples are
not fully ported. Examples for stable Tokio can be found
[here](https://github.com/tokio-rs/tokio/tree/v0.1.x/tokio/examples).
## Getting Help
First, see if the answer to your question can be found in the [Guides] or the
[API documentation]. If the answer is not there, there is an active community in
the [Tokio Gitter channel][chat]. We would be happy to try to answer your
question. Last, if that doesn't work, try opening an [issue] with the question.
[Guides]: https://tokio.rs/docs/
[API documentation]: https://docs.rs/tokio/0.2.0-alpha.2/tokio
[chat]: https://gitter.im/tokio-rs/tokio
[issue]: https://github.com/tokio-rs/tokio/issues/new
## Contributing
:balloon: Thanks for your help improving the project! We are so happy to have
you! We have a [contributing guide][guide] to help you get involved in the Tokio
project.
[guide]: CONTRIBUTING.md
## Project layout
@@ -107,34 +126,62 @@ have greater guarantees of stability.
The crates included as part of Tokio are:
* [`tokio-executor`]: Task execution related traits and utilities.
* [`tokio-executor`]: Task executors and related utilities. Includes a
single-threaded executor and a multi-threaded, work-stealing, executor.
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
* [`tokio-macros`]: Macros for usage with Tokio.
* [`tokio-threadpool`]: Schedules the execution of futures across a pool of
threads.
* [`tokio-net`]: Event loop that drives I/O resources as well as TCP, UDP, and
unix domain socket apis.
* [ `tokio-timer`]: Time related APIs.
[`tokio-codec`]: tokio-codec
[`tokio-current-thread`]: tokio-current-thread
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-io`]: tokio-io
[`tokio-reactor`]: tokio-reactor
[`tokio-threadpool`]: tokio-threadpool
[`tokio-macros`]: tokio-macros
[`tokio-net`]: tokio-net
[`tokio-timer`]: tokio-timer
## Related Projects
In addition to the crates in this repository, the Tokio project also maintains
several other libraries, including:
* [`tracing`] (formerly `tokio-trace`): A framework for application-level
tracing and async-aware diagnostics.
* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers
`tokio`.
* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.
[`tracing`]: https://github.com/tokio-rs/tracing
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Supported Rust Versions
Tokio is built against the latest stable, nightly, and beta Rust releases. The
minimum version supported is the stable release from three months before the
current stable release version. For example, if the latest stable Rust is 1.29,
the minimum version supported is 1.26. The current Tokio version is not
guaranteed to build on Rust versions earlier than the minimum supported version.
## License
This project is licensed under either of
* Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](LICENSE-MIT) or
http://opensource.org/licenses/MIT)
at your option.
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in tokio by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+132
View File
@@ -0,0 +1,132 @@
trigger: ["master", "std-future"]
pr: ["master", "std-future"]
variables:
nightly: nightly-2019-08-21
RUSTFLAGS: -Dwarnings
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
rust: $(nightly)
name: rustfmt
# Apply clippy lints to all crates
- template: ci/azure-clippy.yml
parameters:
rust: $(nightly)
name: clippy
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
rust: $(nightly)
displayName: Test tokio
cross: true
crates:
tokio:
- codec
- fs
- io
- rt-full
- net
- sync
- tcp
- timer
- udp
- uds
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates (cross) -
cross: true
rust: $(nightly)
crates:
tokio-fs: []
tokio-net:
- process
- signal
- tcp
- udp
- uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
rust: $(nightly)
crates:
tokio-buf: []
tokio-codec: []
tokio-executor:
- current-thread
- threadpool
tokio-io:
- util
tokio-sync:
- async-traits
tokio-macros: []
tokio-timer:
- async-traits
tokio-test: []
# Test compilation failure
- template: ci/azure-test-stable.yml
parameters:
name: test_features
displayName: Test feature flags
rust: $(nightly)
crates:
build-tests:
- tokio-executor
- tokio-net
- executor-without-current-thread
- net-no-features
- net-with-tcp
- net-with-udp
- net-with-uds
- tokio-no-features
- tokio-with-net
# # Try cross compiling
# - template: ci/azure-cross-compile.yml
# parameters:
# name: cross_32bit_linux
# target: i686-unknown-linux-gnu
#
# # This represents the minimum Rust version supported by
# # Tokio. Updating this should be done in a dedicated PR and
# # cannot be greater than two 0.x releases prior to the
# # current stable.
# #
# # Tests are not run as tests may require newer versions of
# # rust.
# - template: ci/azure-check-minrust.yml
# parameters:
# name: minrust
# rust_version: 1.34.0
#
# - template: ci/azure-tsan.yml
# parameters:
# name: tsan
# rust: $(nightly)
- template: ci/azure-deploy-docs.yml
parameters:
rust: $(nightly)
dependsOn:
- rustfmt
- clippy
- test_tokio
- test_sub_cross
- test_linux
- test_features
# - test_nightly
# - cross_32bit_linux
# - minrust
# - tsan
Executable
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
set -e
USAGE="Publish a new release of a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-v, --verbose Use verbose Cargo output
-d, --dry-run Perform a dry run (do not publish or tag the release)
-h, --help Show this help text and exit"
DRY_RUN=""
VERBOSE=""
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
verify() {
status "Verifying" "if $CRATE v$VERSION can be released"
ACTUAL=$(cargo pkgid | sed -n 's/.*#\(.*\)/\1/p')
if [ "$ACTUAL" != "$VERSION" ]; then
err "expected to release version $VERSION, but Cargo.toml contained $ACTUAL"
exit 1
fi
if git tag -l | grep -Fxq "$TAG" ; then
err "git tag \`$TAG\` already exists"
exit 1
fi
PATH_DEPS=$(grep -F "path = \"" Cargo.toml | sed -e 's/^/ /')
if [ -n "$PATH_DEPS" ]; then
err "crate \`$CRATE\` contained path dependencies:\n$PATH_DEPS"
echo "path dependencies must be removed prior to release"
exit 1
fi
}
release() {
status "Releasing" "$CRATE v$VERSION"
cargo package $VERBOSE
cargo publish $VERBOSE $DRY_RUN
status "Tagging" "$TAG"
if [ -n "$DRY_RUN" ]; then
echo "# git tag $TAG && git push --tags"
else
git tag "$TAG" && git push --tags
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-v|--verbose)
VERBOSE="--verbose"
set +x
shift
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
(cd "$CRATE" && verify && release )
else
err "no such crate \"$CRATE\""
exit 1
fi
Executable
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
set -e
USAGE="Update links to docs.rs in a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-d, --dry-run Perform a dry run (do not modify any file)
-h, --help Show this help text and exit"
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
c1grep() { grep "$@" || test $? = 1; }
update_versions_in_doc() {
# Print what is being/would be done
if [ -n "$DRY_RUN" ]; then
local MSG="Would change:"
else
local MSG="Updating:"
fi
git grep -lr "docs.rs/$CRATE/" \
| xargs sed --quiet \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|gp" \
| sed -e "s/^/$MSG /"
# Apply changes if not in dry run
if [ -z "$DRY_RUN" ]; then
git grep -lr "docs.rs/$CRATE/" \
| xargs sed -i \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|g"
fi
}
update() {
update_versions_in_doc
}
show_outdated() {
OUTDATED=$(git grep -rn "docs.rs/$CRATE/" \
| c1grep -v "$VERSION" \
| sed -e 's/^/ - /')
if [[ -n "$OUTDATED" ]]; then
echo "Found the following links to docs.rs with an outdated version:"
echo "$OUTDATED"
echo
else
echo "Nothing to do."
exit 1
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
# Does not cd in order to update everywhere
show_outdated && update
else
err "no such crate \"$CRATE\""
exit 1
fi
+25
View File
@@ -0,0 +1,25 @@
[package]
name = "build-tests"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[features]
executor-without-current-thread = ["tokio-executor"]
net-no-features = ["tokio-net"]
net-with-tcp = ["tokio-net/tcp"]
net-with-udp = ["tokio-net/udp"]
net-with-uds = ["tokio-net/uds"]
tokio-no-features = ["tokio"]
tokio-with-net = ["tokio/net"]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
tokio-executor = { path = "../tokio-executor", optional = true }
tokio-net = { path = "../tokio-net", optional = true }
tokio = { path = "../tokio", optional = true, default-features = false }
[dev-dependencies]
trybuild = "1.0"
+2
View File
@@ -0,0 +1,2 @@
Tests the various combination of feature flags. This is broken out to a separate
crate to work around limitations with cargo features.
+8
View File
@@ -0,0 +1,8 @@
#[cfg(feature = "tokio-executor")]
pub use tokio_executor;
#[cfg(feature = "tokio-net")]
pub use tokio_net;
#[cfg(feature = "tokio")]
pub use tokio;
@@ -0,0 +1,3 @@
use build_tests::tokio_executor::current_thread;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_executor::current_thread`
--> $DIR/executor_without_current_thread.rs:1:5
|
1 | use build_tests::tokio_executor::current_thread;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `current_thread` in `tokio_executor`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,4 @@
use build_tests::tokio_net::tcp;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_net::tcp`
--> $DIR/net_without_tcp_missing_tcp.rs:1:5
|
1 | use build_tests::tokio_net::tcp;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `tcp` in `tokio_net`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,4 @@
use build_tests::tokio_net::udp;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_net::udp`
--> $DIR/net_without_udp_missing_udp.rs:1:5
|
1 | use build_tests::tokio_net::udp;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `udp` in `tokio_net`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,4 @@
use build_tests::tokio_net::udp;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio_net::udp`
--> $DIR/net_without_uds_missing_uds.rs:1:5
|
1 | use build_tests::tokio_net::udp;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `udp` in `tokio_net`
For more information about this error, try `rustc --explain E0432`.
@@ -0,0 +1,3 @@
use build_tests::tokio::net;
fn main() {}
@@ -0,0 +1,7 @@
error[E0432]: unresolved import `build_tests::tokio::net`
--> $DIR/tokio_without_net_missing_net.rs:1:5
|
1 | use build_tests::tokio::net;
| ^^^^^^^^^^^^^^^^^^^^^^^ no `net` in `tokio`
For more information about this error, try `rustc --explain E0432`.
+52
View File
@@ -0,0 +1,52 @@
#![allow(unused_imports)]
#[test]
#[cfg(feature = "tokio-net")]
fn net_default() {
use build_tests::tokio_net::driver::{set_default, Handle, Reactor, Registration};
use build_tests::tokio_net::util::PollEvented;
}
#[test]
#[cfg(feature = "net-with-tcp")]
fn net_with_tcp() {
use build_tests::tokio_net::tcp;
}
#[test]
#[cfg(feature = "net-with-udp")]
fn net_with_udp() {
use build_tests::tokio_net::udp;
}
#[test]
#[cfg(feature = "net-with-uds")]
fn net_with_udp() {
use build_tests::tokio_net::uds;
}
#[test]
#[cfg(feature = "tokio-with-net")]
fn tokio_with_net() {
// net is present
use build_tests::tokio::net;
}
#[test]
fn compile_fail() {
let t = trybuild::TestCases::new();
#[cfg(feature = "executor-without-current-thread")]
t.compile_fail("tests/fail/executor_without_current_thread.rs");
#[cfg(feature = "net-no-features")]
{
t.compile_fail("tests/fail/net_without_tcp_missing_tcp.rs");
t.compile_fail("tests/fail/net_without_udp_missing_udp.rs");
t.compile_fail("tests/fail/net_without_uds_missing_uds.rs");
}
#[cfg(feature = "tokio-no-features")]
t.compile_fail("tests/fail/tokio_without_net_missing_net.rs");
drop(t);
}
+29
View File
@@ -0,0 +1,29 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+16
View File
@@ -0,0 +1,16 @@
jobs:
- job: ${{ parameters.name }}
displayName: Clippy
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add clippy
cargo clippy --version
displayName: Install clippy
- script: |
cargo clippy --all --all-features -- -A clippy::mutex-atomic
displayName: cargo clippy --all
+27
View File
@@ -0,0 +1,27 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: sudo apt-get update
displayName: "apt-get update"
- script: sudo apt-get install gcc-multilib
displayName: "Install gcc-multilib"
- script: rustup target add ${{ parameters.target }}
displayName: "Add target"
# Always patch
- template: azure-patch-crates.yml
- script: cargo check --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check source
- script: cargo check --tests --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check tests
+39
View File
@@ -0,0 +1,39 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
# rust_version: stable
rust_version: ${{ parameters.rust }}
- script: |
cargo doc --all --no-deps --all-features
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+33
View File
@@ -0,0 +1,33 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain none
export PATH=$PATH:$HOME/.cargo/bin
rustup toolchain install $RUSTUP_TOOLCHAIN
rustup default $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain none
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
rustup toolchain install %RUSTUP_TOOLCHAIN%
rustup default %RUSTUP_TOOLCHAIN%
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustup toolchain list
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%B' | grep -qF '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+17
View File
@@ -0,0 +1,17 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add rustfmt
cargo fmt --version
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+19
View File
@@ -0,0 +1,19 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
# Check benches
- script: cargo check --benches --all
displayName: Check benchmarks
+61
View File
@@ -0,0 +1,61 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
# rust_version: stable
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
# Run with default crate features
- script: cargo test
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
# Run with each specified feature
- ${{ each feature in crate.value }}:
- script: cargo test --no-default-features --features ${{ feature }}
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test --features ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
# Run with default crate features
- script: cargo test
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
# Run with each specified feature
- ${{ each feature in crate.value }}:
- script: cargo test --no-default-features --features ${{ feature }}
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate.key }} - cargo test --features ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-executor --tests --features threadpool
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+14
View File
@@ -0,0 +1,14 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-buf = { path = "tokio-buf" }
tokio-codec = { path = "tokio-codec" }
tokio-executor = { path = "tokio-executor" }
tokio-fs = { path = "tokio-fs" }
tokio-io = { path = "tokio-io" }
tokio-macros = { path = "tokio-macros" }
tokio-net = { path = "tokio-net" }
tokio-sync = { path = "tokio-sync" }
tokio-timer = { path = "tokio-timer" }
tokio-tls = { path = "tokio-tls" }
+39
View File
@@ -0,0 +1,39 @@
# TSAN suppressions file for Tokio
# TSAN does not understand fences and `Arc::drop` is implemented using a fence.
# This causes many false positives.
race:Arc*drop
race:Weak*drop
# `std` mpsc is not used in any Tokio code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
race:std*lang_start
race:drop*std::thread*
# Probably more fences in std.
race:__call_tls_dtors
# The epoch-based GC uses fences.
race:crossbeam_epoch
# Push and steal operations in crossbeam-deque may cause data races, but such
# data races are safe. If a data race happens, the value read by `steal` is
# forgotten and the steal operation is then retried.
race:crossbeam_deque*push
race:crossbeam_deque*steal
# This filters out expected data race in the Treiber stack implementations.
# Treiber stacks are inherently racy. The pop operation will attempt to access
# the "next" pointer on the node it is attempting to pop. However, at this
# point it has not gained ownership of the node and another thread might beat
# it and take ownership of the node first (touching the next pointer). The
# original pop operation will fail due to the ABA guard, but tsan still picks
# up the access on the next pointer.
race:Backup::next_sleeper
race:Backup::set_next_sleeper
race:WorkerEntry::set_next_sleeper
# This ignores a false positive caused by `thread::park()`/`thread::unpark()`.
# See: https://github.com/rust-lang/rust/pull/54806#issuecomment-436193353
race:pthread_cond_destroy
-52
View File
@@ -1,52 +0,0 @@
## Examples of how to use Tokio
This directory contains a number of examples showcasing various capabilities of
the `tokio` crate.
All examples can be executed with:
```
cargo run --example $name
```
A high level description of each example is:
* [`hello_world`](hello_world.rs) - a tiny server that writes "hello world" to
all connected clients and then terminates the connection, should help see how
to create and initialize `tokio`.
* [`echo`](echo.rs) - this is your standard TCP "echo server" which accepts
connections and then echos back any contents that are read from each connected
client.
* [`echo-udp`](echo-udp.rs) - again your standard "echo server", except for UDP
instead of TCP. This will echo back any packets received to the original
sender.
* [`connect`](connect.rs) - this is a `nc`-like clone which can be used to
interact with most other examples. The program creates a TCP connection or UDP
socket to sends all information read on stdin to the remote peer, displaying
any data received on stdout. Often quite useful when interacting with the
various other servers here!
* [`chat`](chat.rs) - this spins up a local TCP server which will broadcast from
any connected client to all other connected clients. You can connect to this
in multiple terminals and use it to chat between the terminals.
* [`chat-combinator`](chat-combinator.rs) - Similar to `chat`, but this uses a
much more functional programming approch using combinators.
* [`proxy`](proxy.rs) - an example proxy server that will forward all connected
TCP clients to the remote address specified when starting the program.
* [`tinyhttp`](tinyhttp.rs) - a tiny HTTP/1.1 server which doesn't support HTTP
request bodies showcasing running on multiple cores, working with futures and
spawning tasks, and finally framing a TCP connection to discrete
request/response objects.
* [`tinydb`](tinyhttp.rs) - an in-memory database which shows sharing state
between all connected clients, notably the key/value store of this database.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
-150
View File
@@ -1,150 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
extern crate futures;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::sync::{Arc, Mutex};
fn main() {
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse().unwrap();
let socket = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
// This is running on the Tokio runtime, so it will be multi-threaded. The
// `Arc<Mutex<...>>` allows state to be shared across the threads.
let connections = Arc::new(Mutex::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr().unwrap();
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
connections.lock().unwrap().insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.lock().unwrap();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn a task to process the connection
tokio::spawn(connection.then(move |_| {
connections.lock().unwrap().remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}));
Ok(())
});
// execute server
tokio::run(srv);
}
-474
View File
@@ -1,474 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This example is explicitly more verbose than it has to be. This is to
//! illustrate more concepts.
//!
//! A chat server for telnet clients. After a telnet client connects, the first
//! line should contain the client's name. After that, all lines send by a
//! client are broadcasted to all other connected clients.
//!
//! Because the client is telnet, lines are delimited by "\r\n".
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
//! You can run the `telnet` command in any number of additional windows.
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
#[macro_use]
extern crate futures;
extern crate bytes;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use std::collections::HashMap;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<Bytes>;
/// Shorthand for the receive half of the message channel.
type Rx = mpsc::UnboundedReceiver<Bytes>;
/// Data that is shared between all peers in the chat server.
///
/// This is the set of `Tx` handles for all connected clients. Whenever a
/// message is received from a client, it is broadcasted to all peers by
/// iterating over the `peers` entries and sending a copy of the message on each
/// `Tx`.
struct Shared {
peers: HashMap<SocketAddr, Tx>,
}
/// The state for each connected client.
struct Peer {
/// Name of the peer.
///
/// When a client connects, the first line sent is treated as the client's
/// name (like alice or bob). The name is used to preface all messages that
/// arrive from the client so that we can simulate a real chat server:
///
/// ```text
/// alice: Hello everyone.
/// bob: Welcome to telnet chat!
/// ```
name: BytesMut,
/// The TCP socket wrapped with the `Lines` codec, defined below.
///
/// This handles sending and receiving data on the socket. When using
/// `Lines`, we can work at the line level instead of having to manage the
/// raw byte operations.
lines: Lines,
/// Handle to the shared chat state.
///
/// This is used to broadcast messages read off the socket to all connected
/// peers.
state: Arc<Mutex<Shared>>,
/// Receive half of the message channel.
///
/// This is used to receive messages from peers. When a message is received
/// off of this `Rx`, it will be written to the socket.
rx: Rx,
/// Client socket address.
///
/// The socket address is used as the key in the `peers` HashMap. The
/// address is saved so that the `Peer` drop implementation can clean up its
/// entry.
addr: SocketAddr,
}
/// Line based codec
///
/// This decorates a socket and presents a line based read / write interface.
///
/// As a user of `Lines`, we can focus on working at the line level. So, we send
/// and receive values that represent entire lines. The `Lines` codec will
/// handle the encoding and decoding as well as reading from and writing to the
/// socket.
#[derive(Debug)]
struct Lines {
/// The TCP socket.
socket: TcpStream,
/// Buffer used when reading from the socket. Data is not returned from this
/// buffer until an entire line has been read.
rd: BytesMut,
/// Buffer used to stage data before writing it to the socket.
wr: BytesMut,
}
impl Shared {
/// Create a new, empty, instance of `Shared`.
fn new() -> Self {
Shared {
peers: HashMap::new(),
}
}
}
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
// Create a channel for this peer
let (tx, rx) = mpsc::unbounded();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
Peer {
name,
lines,
state,
rx,
addr,
}
}
}
/// This is where a connected client is managed.
///
/// A `Peer` is also a future representing completly processing the client.
///
/// When a `Peer` is created, the first line (representing the client's name)
/// has already been read. When the socket closes, the `Peer` future completes.
///
/// While processing, the peer future implementation will:
///
/// 1) Receive messages on its message channel and write them to the socket.
/// 2) Receive messages from the socket and broadcast them to all peers.
///
impl Future for Peer {
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<(), io::Error> {
// Tokio (and futures) use cooperative scheduling without any
// preemption. If a task never yields execution back to the executor,
// then other tasks may be starved.
//
// To deal with this, robust applications should not have any unbounded
// loops. In this example, we will read at most `LINES_PER_TICK` lines
// from the client on each tick.
//
// If the limit is hit, the current task is notified, informing the
// executor to schedule the task again asap.
const LINES_PER_TICK: usize = 10;
// Receive all messages from peers.
for i in 0..LINES_PER_TICK {
// Polling an `UnboundedReceiver` cannot fail, so `unwrap` here is
// safe.
match self.rx.poll().unwrap() {
Async::Ready(Some(v)) => {
// Buffer the line. Once all lines are buffered, they will
// be flushed to the socket (right below).
self.lines.buffer(&v);
// If this is the last iteration, the loop will break even
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
task::current().notify();
}
}
_ => break,
}
}
// Flush the write buffer to the socket
let _ = self.lines.poll_flush()?;
// Read new lines from the socket
while let Async::Ready(line) = self.lines.poll()? {
println!("Received line ({:?}) : {:?}", self.name, line);
if let Some(message) = line {
// Append the peer's name to the front of the line:
let mut line = self.name.clone();
line.put(": ");
line.put(&message);
line.put("\r\n");
// We're using `Bytes`, which allows zero-copy clones (by
// storing the data in an Arc internally).
//
// However, before cloning, we must freeze the data. This
// converts it from mutable -> immutable, allowing zero copy
// cloning.
let line = line.freeze();
// Now, send the line to all other peers
for (addr, tx) in &self.state.lock().unwrap().peers {
// Don't send the message to ourselves
if *addr != self.addr {
// The send only fails if the rx half has been dropped,
// however this is impossible as the `tx` half will be
// removed from the map before the `rx` is dropped.
tx.unbounded_send(line.clone()).unwrap();
}
}
} else {
// EOF was reached. The remote client has disconnected. There is
// nothing more to do.
return Ok(Async::Ready(()));
}
}
// As always, it is important to not just return `NotReady` without
// ensuring an inner future also returned `NotReady`.
//
// We know we got a `NotReady` from either `self.rx` or `self.lines`, so
// the contract is respected.
Ok(Async::NotReady)
}
}
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
}
}
impl Lines {
/// Create a new `Lines` codec backed by the socket
fn new(socket: TcpStream) -> Self {
Lines {
socket,
rd: BytesMut::new(),
wr: BytesMut::new(),
}
}
/// Buffer a line.
///
/// This writes the line to an internal buffer. Calls to `poll_flush` will
/// attempt to flush this buffer to the socket.
fn buffer(&mut self, line: &[u8]) {
// Ensure the buffer has capacity. Ideally this would not be unbounded,
// but to keep the example simple, we will not limit this.
self.wr.reserve(line.len());
// Push the line onto the end of the write buffer.
//
// The `put` function is from the `BufMut` trait.
self.wr.put(line);
}
/// Flush the write buffer to the socket
fn poll_flush(&mut self) -> Poll<(), io::Error> {
// As long as there is buffered data to write, try to write it.
while !self.wr.is_empty() {
// Try to read some bytes from the socket
let n = try_ready!(self.socket.poll_write(&self.wr));
// As long as the wr is not empty, a successful write should
// never write 0 bytes.
assert!(n > 0);
// This discards the first `n` bytes of the buffer.
let _ = self.wr.split_to(n);
}
Ok(Async::Ready(()))
}
/// Read data from the socket.
///
/// This only returns `Ready` when the socket has closed.
fn fill_read_buf(&mut self) -> Poll<(), io::Error> {
loop {
// Ensure the read buffer has capacity.
//
// This might result in an internal allocation.
self.rd.reserve(1024);
// Read data into the buffer.
let n = try_ready!(self.socket.read_buf(&mut self.rd));
if n == 0 {
return Ok(Async::Ready(()));
}
}
}
}
impl Stream for Lines {
type Item = BytesMut;
type Error = io::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
// First, read any new data that might have been received off the socket
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
if let Some(pos) = pos {
// Remove the line from the read buffer and set it to `line`.
let mut line = self.rd.split_to(pos + 2);
// Drop the trailing \r\n
line.split_off(pos);
// Return the line
return Ok(Async::Ready(Some(line)));
}
if sock_closed {
Ok(Async::Ready(None))
} else {
Ok(Async::NotReady)
}
}
}
/// Spawn a task to manage the socket.
///
/// This will read the first line from the socket to identify the client, then
/// add the client to the set of connected peers in the chat service.
fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// Wrap the socket with the `Lines` codec that we wrote above.
//
// By doing this, we can operate at the line level instead of doing raw byte
// manipulation.
let lines = Lines::new(socket);
// The first line is treated as the client's name. The client is not added
// to the set of connected peers until this line is received.
//
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
// Process the first received line as the client's name.
.and_then(|(name, lines)| {
// If `name` is `None`, then the client disconnected without
// actually sending a line of data.
//
// Since the connection is closed, there is no further work that we
// need to do. So, we just terminate processing by returning
// `future::ok()`.
//
// The problem is that only a single future type can be returned
// from a combinator closure, but we want to return both
// `future::ok()` and `Peer` (below).
//
// This is a common problem, so the `futures` crate solves this by
// providing the `Either` helper enum that allows creating a single
// return type that covers two concrete future types.
let name = match name {
Some(name) => name,
None => {
// The remote client closed the connection without sending
// any data.
return Either::A(future::ok(()));
}
};
println!("`{:?}` is joining the chat", name);
// Create the peer.
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
})
// Task futures have an error of type `()`, this ensures we handle the
// error. We do this by printing the error to STDOUT.
.map_err(|e| {
println!("connection error = {:?}", e);
});
// Spawn the task. Internally, this submits the task to a thread pool.
tokio::spawn(connection);
}
pub fn main() {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse().unwrap();
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
}
-239
View File
@@ -1,239 +0,0 @@
//! An example of hooking up stdin/stdout to either a TCP or UDP stream.
//!
//! This example will connect to a socket address specified in the argument list
//! and then forward all data read on stdin to the server, printing out all data
//! received on stdout. An optional `--udp` argument can be passed to specify
//! that the connection should be made over UDP instead of TCP, translating each
//! line entered on stdin to a UDP packet to be sent to the remote address.
//!
//! Note that this is not currently optimized for performance, especially
//! around buffer management. Rather it's intended to show an example of
//! working with a client.
//!
//! This example can be quite useful when interacting with the other examples in
//! this repository! Many of them recommend running this as a simple "hook up
//! stdin/stdout to a server" to get up and running.
#![deny(warnings)]
extern crate tokio;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use std::env;
use std::io::{self, Read, Write};
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
fn main() {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
Some(i) => {
args.remove(i);
false
}
None => true,
};
// Parse what address we're going to connect to
let addr = args.first().unwrap_or_else(|| {
panic!("this program requires at least one argument")
});
let addr = addr.parse::<SocketAddr>().unwrap();
// Right now Tokio doesn't support a handle to stdin running on the event
// loop, so we farm out that work to a separate thread. This thread will
// read data (with blocking I/O) from stdin and then send it to the event
// loop over a standard futures channel.
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(|| read_stdin(stdin_tx));
let stdin_rx = stdin_rx.map_err(|_| panic!()); // errors not possible on rx
// Now that we've got our stdin read we either set up our TCP connection or
// our UDP connection to get a stream of bytes we're going to emit to
// stdout.
let stdout = if tcp {
tcp::connect(&addr, Box::new(stdin_rx))
} else {
udp::connect(&addr, Box::new(stdin_rx))
};
// And now with our stream of bytes to write to stdout, we execute that in
// the event loop! Note that this is doing blocking I/O to emit data to
// stdout, and in general it's a no-no to do that sort of work on the event
// loop. In this case, though, we know it's ok as the event loop isn't
// otherwise running anything useful.
let mut out = io::stdout();
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
})
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
}
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio_io::codec::{Encoder, Decoder};
/// A simple `Codec` implementation that just ships bytes around.
///
/// This type is used for "framing" a TCP/UDP stream of bytes but it's really
/// just a convenient method for us to work with streams/sinks for now.
/// This'll just take any data read and interpret it as a "frame" and
/// conversely just shove data into the output location without looking at
/// it.
pub struct Bytes;
impl Decoder for Bytes {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() > 0 {
let len = buf.len();
Ok(Some(buf.split_to(len)))
} else {
Ok(None)
}
}
}
impl Encoder for Bytes {
type Item = Vec<u8>;
type Error = io::Error;
fn encode(&mut self, data: Vec<u8>, buf: &mut BytesMut) -> io::Result<()> {
buf.put(&data[..]);
Ok(())
}
}
}
mod tcp {
use tokio;
use tokio::net::TcpStream;
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
// to start forwarding data.
//
// First we use the `Io::framed` method with a simple implementation of
// a `Codec` (listed below) that just ships bytes around. We then split
// that in two to work with the stream and sink separately.
//
// Half of the work we're going to do is to take all data we receive on
// `stdin` and send that along the TCP stream (`sink`). The second half
// is to take all the data we receive (`stream`) and then write that to
// stdout. We'll be passing this handle back out from this method.
//
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
Box::new(tcp.map(move |stream| {
let (sink, stream) = stream.framed(Bytes).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream())
}
}
mod udp {
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
"0.0.0.0:0".parse().unwrap()
} else {
"[::]:0".parse().unwrap()
};
let udp = UdpSocket::bind(&addr_to_bind)
.expect("failed to bind socket");
// Like above with TCP we use an instance of `Bytes` codec to transform
// this UDP socket into a framed sink/stream which operates over
// discrete values. In this case we're working with *pairs* of socket
// addresses and byte buffers.
let (sink, stream) = UdpFramed::new(udp, Bytes).split();
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
tokio::spawn(stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
Box::new(stream.filter_map(move |(chunk, src)| {
if src == addr {
Some(chunk.into())
} else {
None
}
}))
}
}
// Our helper method which will read data from stdin and send it along the
// sender provided.
fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
let mut stdin = io::stdin();
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
tx = match tx.send(buf).wait() {
Ok(tx) => tx,
Err(_) => break,
};
}
}
-114
View File
@@ -1,114 +0,0 @@
//! A "hello world" echo server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! write back everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread poool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example echo
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be echo'd back to
//! you! If you open up multiple terminals running the `connect` example you
//! should be able to see them all make progress simultaneously.
#![deny(warnings)]
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
fn main() {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We just want to copy all data read from the socket back onto the
// socket itself (e.g. "echo"). We can use the standard `io::copy`
// combinator in the `tokio-core` crate to do precisely this!
//
// The `copy` function takes two arguments, where to read from and where
// to write to. We only have one argument, though, with `socket`.
// Luckily there's a method, `Io::split`, which will split an Read/Write
// stream into its two halves. This operation allows us to work with
// each stream independently, such as pass them as two arguments to the
// `copy` function.
//
// The `copy` function then returns a future, and this future will be
// resolved when the copying operation is complete, resolving to the
// amount of data that was copied.
let (reader, writer) = socket.split();
let amt = io::copy(reader, writer);
// After our copy operation is complete we just print out some helpful
// information.
let msg = amt.then(move |result| {
match result {
Ok((amt, _, _)) => println!("wrote {} bytes", amt),
Err(e) => println!("error: {}", e),
}
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(msg)
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
}
-70
View File
@@ -1,70 +0,0 @@
//! Hello world server.
//!
//! A simple server that accepts connections, writes "hello world\n", and closes
//! the connection.
//!
//! You can test this out by running:
//!
//! cargo run --example hello_world
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
#![deny(warnings)]
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
pub fn main() {
let addr = "127.0.0.1:6142".parse().unwrap();
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(|socket| {
println!("accepted socket; addr={:?}", socket.peer_addr().unwrap());
let connection = io::write_all(socket, "hello world\n")
.then(|res| {
println!("wrote message; success={:?}", res.is_ok());
Ok(())
});
// Spawn a new task that processes the socket:
tokio::spawn(connection);
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
}
-128
View File
@@ -1,128 +0,0 @@
//! A proxy that forwards data to another server and forwards that server's
//! responses back to clients.
//!
//! Because the Tokio runtime uses a thread poool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! You can showcase this by running this in one terminal:
//!
//! cargo run --example proxy
//!
//! This in another terminal
//!
//! cargo run --example echo
//!
//! And finally this in another terminal
//!
//! cargo run --example connect 127.0.0.1:8081
//!
//! This final terminal will connect to our proxy, which will in turn connect to
//! the echo server, and you'll be able to see data flowing between them.
#![deny(warnings)]
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use tokio::io::{copy, shutdown};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
fn main() {
let listen_addr = env::args().nth(1).unwrap_or("127.0.0.1:8081".to_string());
let listen_addr = listen_addr.parse::<SocketAddr>().unwrap();
let server_addr = env::args().nth(2).unwrap_or("127.0.0.1:8080".to_string());
let server_addr = server_addr.parse::<SocketAddr>().unwrap();
// Create a TCP listener which will listen for incoming connections.
let socket = TcpListener::bind(&listen_addr).unwrap();
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
let amounts = server.and_then(move |server| {
// Create separate read/write handles for the TCP clients that we're
// proxying data between. Note that typically you'd use
// `AsyncRead::split` for this operation, but we want our writer
// handles to have a custom implementation of `shutdown` which
// actually calls `TcpStream::shutdown` to ensure that EOF is
// transmitted properly across the proxied connection.
//
// As a result, we wrap up our client/server manually in arcs and
// use the impls below on our custom `MyTcpStream` type.
let client_reader = MyTcpStream(Arc::new(Mutex::new(client)));
let client_writer = client_reader.clone();
let server_reader = MyTcpStream(Arc::new(Mutex::new(server)));
let server_writer = server_reader.clone();
// Copy the data (in parallel) between the client and the server.
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
Ok(())
});
tokio::run(done);
}
// This is a custom type used to have a custom implementation of the
// `AsyncWrite::shutdown` method which actually calls `TcpStream::shutdown` to
// notify the remote end that we're done writing.
#[derive(Clone)]
struct MyTcpStream(Arc<Mutex<TcpStream>>);
impl Read for MyTcpStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.0.lock().unwrap().read(buf)
}
}
impl Write for MyTcpStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().unwrap().write(buf)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl AsyncRead for MyTcpStream {}
impl AsyncWrite for MyTcpStream {
fn shutdown(&mut self) -> Poll<(), io::Error> {
try!(self.0.lock().unwrap().shutdown(Shutdown::Write));
Ok(().into())
}
}
-206
View File
@@ -1,206 +0,0 @@
//! A "tiny database" and accompanying protocol
//!
//! This example shows the usage of shared state amongst all connected clients,
//! namely a database of key/value pairs. Each connected client can send a
//! series of GET/SET commands to query the current value of a key or set the
//! value of a key.
//!
//! This example has a simple protocol you can use to interact with the server.
//! To run, first run this in one terminal window:
//!
//! cargo run --example tinydb
//!
//! and next in another windows run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! In the `connect` window you can type in commands where when you hit enter
//! you'll get a response from the server for that command. An example session
//! is:
//!
//!
//! $ cargo run --example connect 127.0.0.1:8080
//! GET foo
//! foo = bar
//! GET FOOBAR
//! error: no key FOOBAR
//! SET FOOBAR my awesome string
//! set FOOBAR = `my awesome string`, previous: None
//! SET foo tokio
//! set foo = `tokio`, previous: Some("bar")
//! GET foo
//! foo = tokio
//!
//! Namely you can issue two forms of commands:
//!
//! * `GET $key` - this will fetch the value of `$key` from the database and
//! return it. The server's database is initially populated with the key `foo`
//! set to the value `bar`
//! * `SET $key $value` - this will set the value of `$key` to `$value`,
//! returning the previous value, if any.
#![deny(warnings)]
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use tokio::io::{lines, write_all};
use tokio::net::TcpListener;
use tokio::prelude::*;
/// The in-memory database shared amongst all clients.
///
/// This database will be shared via `Arc`, so to mutate the internal map we're
/// also going to use a `RefCell` for interior mutability.
struct Database {
map: Mutex<HashMap<String, String>>,
}
/// Possible requests our clients can send us
enum Request {
Get { key: String },
Set { key: String, value: String },
}
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
}
fn main() {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let listener = TcpListener::bind(&addr).expect("failed to bind");
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
// clients. We populate the initial database and then create the `Database`
// structure. Note the usage of `Arc` here which will be used to ensure that
// each independently spawned client will have a reference to the in-memory
// database.
let mut initial_db = HashMap::new();
initial_db.insert("foo".to_string(), "bar".to_string());
let db = Arc::new(Database {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
// *split* this TCP stream into two separately owned halves. This'll
// allow us to work with the read and write halves independently.
let (reader, writer) = socket.split();
// Since our protocol is line-based we use `tokio_io`'s `lines` utility
// to convert our stream of bytes, `reader`, into a `Stream` of lines.
let lines = lines(BufReader::new(reader));
// Here's where the meat of the processing in this server happens. First
// we see a clone of the database being created, which is creating a
// new reference for this connected client to use. Also note the `move`
// keyword on the closure here which moves ownership of the reference
// into the closure, which we'll need for spawning the client below.
//
// The `map` function here means that we'll run some code for all
// requests (lines) we receive from the client. The actual handling here
// is pretty simple, first we parse the request and if it's valid we
// generate a response based on the values in the database.
let db = db.clone();
let responses = lines.map(move |line| {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
}
}
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
}
}
});
// At this point `responses` is a stream of `Response` types which we
// now want to write back out to the client. To do that we use
// `Stream::fold` to perform a loop here, serializing each response and
// then writing it out to the client.
let writes = responses.fold(writer, |writer, response| {
let mut response = response.serialize();
response.push('\n');
write_all(writer, response.into_bytes()).map(|(w, _)| w)
});
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients, for now ignoring any errors
// that we see.
let msg = writes.then(move |_| Ok(()));
tokio::spawn(msg)
});
tokio::run(done);
}
impl Request {
fn parse(input: &str) -> Result<Request, String> {
let mut parts = input.splitn(3, " ");
match parts.next() {
Some("GET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err(format!("GET must be followed by a key")),
};
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
}
Ok(Request::Get { key: key.to_string() })
}
Some("SET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err(format!("SET must be followed by a key")),
};
let value = match parts.next() {
Some(value) => value,
None => return Err(format!("SET needs a value")),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
}
}
}
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
}
}
}
-63
View File
@@ -1,63 +0,0 @@
//! This example leverages `BytesCodec` to create a UDP client and server which
//! speak a custom protocol.
//!
//! Here we're using the codec from tokio-io to convert a UDP socket to a stream of
//! client messages. These messages are then processed and returned back as a
//! new message with a new destination. Overall, we then use this to construct a
//! "ping pong" pair where two sockets are sending messages back and forth.
#![deny(warnings)]
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::net::SocketAddr;
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_io::codec::BytesCodec;
fn main() {
let _ = env_logger::init();
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr).unwrap();
let b = UdpSocket::bind(&addr).unwrap();
let b_addr = b.local_addr().unwrap();
// We're parsing each socket with the `LineCodec` defined above, and then we
// `split` each codec into the sink/stream halves.
let (a_sink, a_stream) = UdpFramed::new(a, BytesCodec::new()).split();
let (b_sink, b_stream) = UdpFramed::new(b, BytesCodec::new()).split();
// Start off by sending a ping from a to b, afterwards we just print out
// what they send us and continually send pings
// let pings = stream::iter((0..5).map(Ok));
let a = a_sink.send(("PING".into(), b_addr)).and_then(|a_sink| {
let mut i = 0;
let a_stream = a_stream.take(4).map(move |(msg, addr)| {
i += 1;
println!("[a] recv: {}", String::from_utf8_lossy(&msg));
(format!("PING {}", i).into(), addr)
});
a_sink.send_all(a_stream)
});
// The second client we have will receive the pings from `a` and then send
// back pongs.
let b_stream = b_stream.map(|(msg, addr)| {
println!("[b] recv: {}", String::from_utf8_lossy(&msg));
("PONG".into(), addr)
});
let b = b_sink.send_all(b_stream);
// Spawn the sender of pongs and then wait for our pinger to finish.
tokio::run({
b.join(a)
.map(|_| ())
.map_err(|e| println!("error = {:?}", e))
});
}
+1
View File
@@ -0,0 +1 @@
nightly-2019-08-21
-713
View File
@@ -1,713 +0,0 @@
//! Execute many tasks concurrently on the current thread.
//!
//! [`CurrentThread`] is an executor that keeps tasks on the same thread that
//! they were spawned from. This allows it to execute futures that are not
//! `Send`.
//!
//! A single [`CurrentThread`] instance is able to efficiently manage a large
//! number of tasks and will attempt to schedule all tasks fairly.
//!
//! All tasks that are being managed by a [`CurrentThread`] executor are able to
//! spawn additional tasks by calling [`spawn`]. This function only works from
//! within the context of a running [`CurrentThread`] instance.
//!
//! The easiest way to start a new [`CurrentThread`] executor is to call
//! [`block_on_all`] with an initial task to seed the executor.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread;
//! use futures::future::lazy;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! current_thread::block_on_all(lazy(|| {
//! // The execution context is setup, futures may be executed.
//! current_thread::spawn(lazy(|| {
//! println!("called from the current thread executor");
//! Ok(())
//! }));
//!
//! Ok::<_, ()>(())
//! }));
//! # }
//! ```
//!
//! The `block_on_all` function will block the current thread until **all**
//! tasks that have been spawned onto the [`CurrentThread`] instance have
//! completed.
//!
//! More fine-grain control can be achieved by using [`CurrentThread`] directly.
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread::CurrentThread;
//! use futures::future::{lazy, empty};
//! use std::time::Duration;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! let mut current_thread = CurrentThread::new();
//!
//! // Spawn a task, the task is not executed yet.
//! current_thread.spawn(lazy(|| {
//! println!("Spawning a task");
//! Ok(())
//! }));
//!
//! // Spawn a task that never completes
//! current_thread.spawn(empty());
//!
//! // Run the executor, but only until the provided future completes. This
//! // provides the opportunity to start executing previously spawned tasks.
//! let res = current_thread.block_on(lazy(|| {
//! Ok::<_, ()>("Hello")
//! })).unwrap();
//!
//! // Now, run the executor for *at most* 1 second. Since a task was spawned
//! // that never completes, this function will return with an error.
//! current_thread.run_timeout(Duration::from_secs(1)).unwrap_err();
//! # }
//! ```
//!
//! # Execution model
//!
//! Internally, [`CurrentThread`] maintains a queue. When one of its tasks is
//! notified, the task gets added to the queue. The executor will pop tasks from
//! the queue and call [`Future::poll`]. If the task gets notified while it is
//! being executed, it won't get re-executed until all other tasks currently in
//! the queue get polled.
//!
//! Before the task is polled, a thread-local variable referencing the current
//! [`CurrentThread`] instance is set. This enables [`spawn`] to spawn new tasks
//! onto the same executor without having to thread through a handle value.
//!
//! If the [`CurrentThread`] instance still has uncompleted tasks, but none of
//! these tasks are ready to be polled, the current thread is put to sleep. When
//! a task is notified, the thread is woken up and processing resumes.
//!
//! All tasks managed by [`CurrentThread`] remain on the current thread. When a
//! task completes, it is dropped.
//!
//! [`spawn`]: fn.spawn.html
//! [`block_on_all`]: fn.block_on_all.html
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
#![allow(deprecated)]
mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::{self, Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::{executor, Async, Future};
use futures::future::{self, Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::marker::PhantomData;
use std::rc::Rc;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
/// Execute futures and receive unpark notifications.
scheduler: Scheduler<P::Unpark>,
/// Current number of futures being executed
num_futures: usize,
/// Thread park handle
park: P,
}
/// Executes futures on the current thread.
///
/// All futures executed using this executor will be executed on the current
/// thread. As such, `run` will wait for these futures to complete before
/// returning.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
// Prevent the handle from moving across threads.
_p: ::std::marker::PhantomData<Rc<()>>,
}
/// Returned by the `turn` function
#[derive(Debug)]
pub struct Turn(());
/// A `CurrentThread` instance bound to a supplied execution conext.
pub struct Entered<'a, P: Park + 'a> {
executor: &'a mut CurrentThread<P>,
enter: &'a mut Enter,
}
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[derive(Debug)]
pub struct Context<'a> {
cancel: Cell<bool>,
_p: PhantomData<&'a ()>,
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
_p: (),
}
/// Error returned by the `run_timeout` function.
#[derive(Debug)]
pub struct RunTimeoutError {
timeout: bool,
}
/// Error returned by the `turn` function.
#[derive(Debug)]
pub struct TurnError {
_p: (),
}
/// Error returned by the `block_on` function.
#[derive(Debug)]
pub struct BlockError<T> {
inner: Option<T>,
}
/// This is mostly split out to make the borrow checker happy.
struct Borrow<'a, U: 'a> {
scheduler: &'a mut Scheduler<U>,
num_futures: &'a mut usize,
}
trait SpawnLocal {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>);
}
struct CurrentRunner {
spawn: Cell<Option<*mut SpawnLocal>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
});
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[allow(deprecated)]
pub fn run<F, R>(f: F) -> R
where F: FnOnce(&mut Context) -> R
{
let mut context = Context {
cancel: Cell::new(false),
_p: PhantomData,
};
let mut current_thread = CurrentThread::new();
let ret = current_thread
.block_on(future::lazy(|| Ok::<_, ()>(f(&mut context))))
.unwrap();
if context.cancel.get() {
return ret;
}
current_thread.run().unwrap();
ret
}
/// Run the executor bootstrapping the execution with the provided future.
///
/// This creates a new [`CurrentThread`] executor, spawns the provided future,
/// and blocks the current thread until the provided future and **all**
/// subsequently spawned futures complete. In other words:
///
/// * If the provided boostrap future does **not** spawn any additional tasks,
/// `block_on_all` returns once `future` completes.
/// * If the provided bootstrap future **does** spawn additional tasks, then
/// `block_on_all` returns once **all** spawned futures complete.
///
/// See [module level][mod] documentation for more details.
///
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
{
let mut current_thread = CurrentThread::new();
let ret = current_thread.block_on(future);
current_thread.run().unwrap();
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
}
/// Executes a future on the current thread.
///
/// The provided future must complete or be canceled before `run` will return.
///
/// Unlike [`tokio::spawn`], this function will always spawn on a
/// `CurrentThread` executor and is able to spawn futures that are not `Send`.
///
/// # Panics
///
/// This function can only be invoked from the context of a `run` call; any
/// other use will result in a panic.
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
{
TaskExecutor::current()
.spawn_local(Box::new(future))
.unwrap();
}
// ===== impl CurrentThread =====
impl CurrentThread<ParkThread> {
/// Create a new instance of `CurrentThread`.
pub fn new() -> Self {
CurrentThread::new_with_park(ParkThread::new())
}
}
impl<P: Park> CurrentThread<P> {
/// Create a new instance of `CurrentThread` backed by the given park
/// handle.
pub fn new_with_park(park: P) -> Self {
let unpark = park.unpark();
CurrentThread {
scheduler: Scheduler::new(unpark),
num_futures: 0,
park,
}
}
/// Returns `true` if the executor is currently idle.
///
/// An idle executor is defined by not currently having any spawned tasks.
pub fn is_idle(&self) -> bool {
self.num_futures == 0
}
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).turn(duration)
}
/// Bind `CurrentThread` instance with an execution context.
pub fn enter<'a>(&'a mut self, enter: &'a mut Enter) -> Entered<'a, P> {
Entered {
executor: self,
enter,
}
}
fn borrow(&mut self) -> Borrow<P::Unpark> {
Borrow {
scheduler: &mut self.scheduler,
num_futures: &mut self.num_futures,
}
}
}
impl tokio_executor::Executor for CurrentThread {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.borrow().spawn_local(future);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures)
.finish()
}
}
// ===== impl Entered =====
impl<'a, P: Park> Entered<'a, P> {
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
match res {
Ok(Async::Ready(e)) => return Ok(e),
Err(e) => return Err(BlockError { inner: Some(e) }),
Ok(Async::NotReady) => {}
}
self.tick();
if let Err(_) = self.executor.park.park() {
return Err(BlockError { inner: None });
}
}
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
if !self.tick() {
let res = match duration {
Some(duration) => self.executor.park.park_timeout(duration),
None => self.executor.park.park(),
};
if res.is_err() {
return Err(TurnError { _p: () });
}
self.tick();
}
Ok(Turn(()))
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
if self.executor.is_idle() {
// Nothing to do
return Ok(());
}
let mut time = dur.map(|dur| (Instant::now() + dur, dur));
loop {
self.tick();
if self.executor.is_idle() {
return Ok(());
}
match time {
Some((until, rem)) => {
if let Err(_) = self.executor.park.park_timeout(rem) {
return Err(RunTimeoutError::new(false));
}
let now = Instant::now();
if now >= until {
return Err(RunTimeoutError::new(true));
}
time = Some((until, until - now));
}
None => {
if let Err(_) = self.executor.park.park() {
return Err(RunTimeoutError::new(false));
}
}
}
}
}
/// Returns `true` if any futures were processed
fn tick(&mut self) -> bool {
self.executor.scheduler.tick(
&mut *self.enter,
&mut self.executor.num_futures)
}
}
impl<'a, P: Park> fmt::Debug for Entered<'a, P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Entered")
.field("executor", &self.executor)
.field("enter", &self.enter)
.finish()
}
}
// ===== impl TaskExecutor =====
#[deprecated(since = "0.1.2", note = "use TaskExecutor::current instead")]
#[doc(hidden)]
pub fn task_executor() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
impl TaskExecutor {
/// Returns an executor that executes futures on the current thread.
///
/// The user of `TaskExecutor` must ensure that when a future is submitted,
/// that it is done within the context of a call to `run`.
///
/// For more details, see the [module level](index.html) documentation.
pub fn current() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
}
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.spawn_local(future)
}
fn status(&self) -> Result<(), SpawnError> {
CURRENT.with(|current| {
if current.spawn.get().is_some() {
Ok(())
} else {
Err(SpawnError::shutdown())
}
})
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future)) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
}
})
}
}
// ===== impl Context =====
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
// ===== impl Borrow =====
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.set_spawn(self, || {
f()
})
})
}
}
impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>) {
*self.num_futures += 1;
self.scheduler.schedule(future);
}
}
// ===== impl CurrentRunner =====
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
{
struct Reset<'a>(&'a CurrentRunner);
impl<'a> Drop for Reset<'a> {
fn drop(&mut self) {
self.0.spawn.set(None);
}
}
let _reset = Reset(self);
let spawn = unsafe { hide_lt(spawn as *mut SpawnLocal) };
self.spawn.set(Some(spawn));
f()
}
}
unsafe fn hide_lt<'a>(p: *mut (SpawnLocal + 'a)) -> *mut (SpawnLocal + 'static) {
use std::mem;
mem::transmute(p)
}
// ===== impl RunTimeoutError =====
impl RunTimeoutError {
fn new(timeout: bool) -> Self {
RunTimeoutError { timeout }
}
/// Returns `true` if the error was caused by the operation timeing out.
pub fn is_timeout(&self) -> bool {
self.timeout
}
}
impl From<tokio_executor::EnterError> for RunTimeoutError {
fn from(_: tokio_executor::EnterError) -> Self {
RunTimeoutError::new(false)
}
}
// ===== impl BlockError =====
impl<T> BlockError<T> {
/// Returns the error yielded by the future being blocked on
pub fn into_inner(self) -> Option<T> {
self.inner
}
}
impl<T> From<tokio_executor::EnterError> for BlockError<T> {
fn from(_: tokio_executor::EnterError) -> Self {
BlockError { inner: None }
}
}
-217
View File
@@ -1,217 +0,0 @@
//! Task execution utilities.
//!
//! In the Tokio execution model, futures are lazy. When a future is created, no
//! work is performed. In order for the work defined by the future to happen,
//! the future must be submitted to an executor. A future that is submitted to
//! an executor is called a "task".
//!
//! The executor executor is responsible for ensuring that [`Future::poll`] is
//! called whenever the task is [notified]. Notification happens when the
//! internal state of a task transitions from "not ready" to ready. For
//! example, a socket might have received data and a call to `read` will now be
//! able to succeed.
//!
//! The specific strategy used to manage the tasks is left up to the
//! executor. There are two main flavors of executors: single-threaded and
//! multithreaded. This module provides both.
//!
//! * **[`current_thread`]**: A single-threaded executor that support spawning
//! tasks that are not `Send`. It guarantees that tasks will be executed on
//! the same thread from which they are spawned.
//!
//! * **[`thread_pool`]**: A multi-threaded executor that maintains a pool of
//! threads. Tasks are spawned to one of the threads in the pool and executed.
//! The pool employes a [work-stealing] strategy for optimizing how tasks get
//! spread across the available threads.
//!
//! # `Executor` trait.
//!
//! This module provides the [`Executor`] trait (re-exported from
//! [`tokio-executor`]), which describes the API that all executors must
//! implement.
//!
//! A free [`spawn`] function is provided that allows spawning futures onto the
//! default executor (tracked via a thread-local variable) without referencing a
//! handle. It is expected that all executors will set a value for the default
//! executor. This value will often be set to the executor itself, but it is
//! possible that the default executor might be set to a different executor.
//!
//! For example, the [`current_thread`] executor might set the default executor
//! to a thread pool instead of itself, allowing futures to spawn new tasks onto
//! the thread pool when those tasks are `Send`.
//!
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
//! [notified]: https://docs.rs/futures/0.1/futures/executor/trait.Notify.html#tymethod.notify
//! [`current_thread`]: current_thread/index.html
//! [`thread_pool`]: thread_pool/index.html
//! [work-stealing]: https://en.wikipedia.org/wiki/Work_stealing
//! [`tokio-executor`]: #
//! [`Executor`]: #
//! [`spawn`]: #
pub mod current_thread;
pub mod thread_pool {
//! Maintains a pool of threads across which the set of spawned tasks are
//! executed.
//!
//! [`ThreadPool`] is an executor that uses a thread pool for executing
//! tasks concurrently across multiple cores. It uses a thread pool that is
//! optimized for use cases that involve multiplexing large number of
//! independent tasks that perform short(ish) amounts of computation and are
//! mainly waiting on I/O, i.e. the Tokio use case.
//!
//! Usually, users of [`ThreadPool`] will not create pool instances.
//! Instead, they will create a [`Runtime`] instance, which comes with a
//! pre-configured thread pool.
//!
//! At the core, [`ThreadPool`] uses a work-stealing based scheduling
//! strategy. When spawning a task while *external* to the thread pool
//! (i.e., from a thread that is not part of the thread pool), the task is
//! randomly assigned to a worker thread. When spawning a task while
//! *internal* to the thread pool, the task is assigned to the current
//! worker.
//!
//! Each worker maintains its own queue and first focuses on processing all
//! tasks in its queue. When the worker's queue is empty, the worker will
//! attempt to *steal* tasks from other worker queues. This strategy helps
//! ensure that work is evenly distributed across threads while minimizing
//! synchronization between worker threads.
//!
//! # Usage
//!
//! Thread pool instances are created using [`ThreadPool::new`] or
//! [`Builder::new`]. The first option returns a thread pool with default
//! configuration values. The second option allows configuring the thread
//! pool before instantiating it.
//!
//! Once an instance is obtained, futures may be spawned onto it using the
//! [`spawn`] function.
//!
//! A handle to the thread pool is obtained using [`ThreadPool::sender`].
//! This handle is **only** able to spawn futures onto the thread pool. It
//! is unable to affect the lifecycle of the thread pool in any way. This
//! handle can be passed into functions or stored in structs as a way to
//! grant the capability of spawning futures.
//!
//! # Examples
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::thread_pool::ThreadPool;
//! use futures::future::{Future, lazy};
//!
//! # pub fn main() {
//! // Create a thread pool with default configuration values
//! let thread_pool = ThreadPool::new();
//!
//! thread_pool.spawn(lazy(|| {
//! println!("called from a worker thread");
//! Ok(())
//! }));
//!
//! // Gracefully shutdown the threadpool
//! thread_pool.shutdown().wait().unwrap();
//! # }
//! ```
//!
//! [`ThreadPool`]: struct.ThreadPool.html
//! [`ThreadPool::new`]: struct.ThreadPool.html#method.new
//! [`ThreadPool::sender`]: struct.ThreadPool.html#method.sender
//! [`spawn`]: struct.ThreadPool.html#method.spawn
//! [`Builder::new`]: struct.Builder.html#method.new
//! [`Runtime`]: ../../runtime/struct.Runtime.html
pub use tokio_threadpool::{
Builder,
Sender,
Shutdown,
ThreadPool,
};
}
pub use tokio_executor::{Executor, DefaultExecutor, SpawnError};
use futures::{Future, IntoFuture};
use futures::future::{self, FutureResult};
/// Return value from the `spawn` function.
///
/// Currently this value doesn't actually provide any functionality. However, it
/// provides a way to add functionality later without breaking backwards
/// compatibility.
///
/// This also implements `IntoFuture` so that it can be used as the return value
/// in a `for_each` loop.
///
/// See [`spawn`] for more details.
///
/// [`spawn`]: fn.spawn.html
#[derive(Debug)]
pub struct Spawn(());
/// Spawns a future on the default executor.
///
/// In order for a future to do work, it must be spawned on an executor. The
/// `spawn` function is the easiest way to do this. It spawns a future on the
/// [default executor] for the current execution context (tracked using a
/// thread-local variable).
///
/// The default executor is **usually** a thread pool.
///
/// # Examples
///
/// In this example, a server is started and `spawn` is used to start a new task
/// that processes each received connection.
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// [default executor]: struct.DefaultExecutor.html
///
/// # Panics
///
/// This function will panic if the default executor is not set or if spawning
/// onto the default executor returns an error. To avoid the panic, use
/// [`DefaultExecutor`].
///
/// [`DefaultExecutor`]: #
pub fn spawn<F>(f: F) -> Spawn
where F: Future<Item = (), Error = ()> + 'static + Send
{
::tokio_executor::spawn(f);
Spawn(())
}
impl IntoFuture for Spawn {
type Future = FutureResult<(), ()>;
type Item = ();
type Error = ();
fn into_future(self) -> Self::Future {
future::ok(())
}
}
-189
View File
@@ -1,189 +0,0 @@
//! A runtime for writing reliable, asynchronous, and slim applications.
//!
//! Tokio is an event-driven, non-blocking I/O platform for writing asynchronous
//! applications with the Rust programming language. At a high level, it
//! provides a few major components:
//!
//! * A multi threaded, work-stealing based task [scheduler][runtime].
//! * A [reactor][reactor] backed by the operating system's event queue (epoll, kqueue,
//! IOCP, etc...).
//! * Asynchronous [TCP and UDP][net] sockets.
//!
//! Tokio is built using futures (provided by the [futures] crate) as the
//! abstraction for managing the complexity of asynchronous programming.
//!
//! Guide level documentation is found on the [website].
//!
//! [website]: https://tokio.rs/docs/getting-started/hello-world/
//! [futures]: http://docs.rs/futures
//!
//! # Examples
//!
//! A simple TCP echo server:
//!
//! ```no_run
//! extern crate tokio;
//!
//! use tokio::prelude::*;
//! use tokio::io::copy;
//! use tokio::net::TcpListener;
//!
//! fn main() {
//! // Bind the server's socket.
//! let addr = "127.0.0.1:12345".parse().unwrap();
//! let listener = TcpListener::bind(&addr)
//! .expect("unable to bind TCP listener");
//!
//! // Pull out a stream of sockets for incoming connections
//! let server = listener.incoming()
//! .map_err(|e| eprintln!("accept failed = {:?}", e))
//! .for_each(|sock| {
//! // Split up the reading and writing parts of the
//! // socket.
//! let (reader, writer) = sock.split();
//!
//! // A future that echos the data and returns how
//! // many bytes were copied...
//! let bytes_copied = copy(reader, writer);
//!
//! // ... after which we'll print what happened.
//! let handle_conn = bytes_copied.map(|amt| {
//! println!("wrote {:?} bytes", amt)
//! }).map_err(|err| {
//! eprintln!("IO error {:?}", err)
//! });
//!
//! // Spawn the future as a concurrent task.
//! tokio::spawn(handle_conn)
//! });
//!
//! // Start the Tokio runtime
//! tokio::run(server);
//! }
//! ```
#![doc(html_root_url = "https://docs.rs/tokio/0.1.3")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
extern crate bytes;
#[macro_use]
extern crate futures;
extern crate iovec;
extern crate mio;
extern crate slab;
extern crate tokio_io;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_threadpool;
#[macro_use]
extern crate log;
pub mod executor;
pub mod net;
pub mod reactor;
pub mod runtime;
pub use executor::spawn;
pub use runtime::run;
pub mod io {
//! Asynchronous I/O.
//!
//! This module is the asynchronous version of `std::io`. Primarily, it
//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which extend the
//! `Read` and `Write` traits of the standard library.
//!
//! [`AsyncRead`] and [`AsyncWrite`] must only be implemented for
//! non-blocking I/O types that integrate with the futures type system. In
//! other words, these types must never block the thread, and instead the
//! current task is notified when the I/O resource is ready.
//!
//! Utilities functions are provided for working with [`AsyncRead`] /
//! [`AsyncWrite`] types. For example, [`copy`] asynchronously copies all
//! data from a source to a destination.
//!
//! Additionally, [`Read`], [`Write`], [`Error`], [`ErrorKind`], and
//! [`Result`] are re-exported from `std::io` for ease of use.
//!
//! [`AsyncRead`]: trait.AsyncRead.html
//! [`AsyncWrite`]: trait.AsyncWrite.html
//! [`copy`]: fn.copy.html
//! [`Read`]: trait.Read.html
//! [`Write`]: trait.Write.html
//! [`Error`]: struct.Error.html
//! [`ErrorKind`]: enum.ErrorKind.html
//! [`Result`]: type.Result.html
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
// Utils
pub use tokio_io::io::{
copy,
Copy,
flush,
Flush,
lines,
Lines,
read_exact,
ReadExact,
read_to_end,
ReadToEnd,
read_until,
ReadUntil,
shutdown,
Shutdown,
write_all,
WriteAll,
};
// Re-export io::Error so that users don't have to deal
// with conflicts when `use`ing `futures::io` and `std::io`.
pub use ::std::io::{
Error,
ErrorKind,
Result,
Read,
Write,
};
}
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
pub use ::std::io::{
Read,
Write,
};
pub use futures::{
Future,
future,
Stream,
stream,
Sink,
IntoFuture,
Async,
AsyncSink,
Poll,
task,
};
}
-44
View File
@@ -1,44 +0,0 @@
//! TCP/UDP bindings for `tokio`.
//!
//! This module contains the TCP/UDP networking types, similar to the standard
//! library, which can be used to implement networking protocols.
//!
//! # TCP
//!
//! Connecting to an address, via TCP, can be done using [`TcpStream`]'s
//! [`connect`] method, which returns [`ConnectFuture`]. `ConnectFuture`
//! implements a future which returns a `TcpStream`.
//!
//! To listen on an address [`TcpListener`] can be used. `TcpListener`'s
//! [`incoming`][incoming_method] method can be used to accept new connections.
//! It return the [`Incoming`] struct, which implements a stream which returns
//! `TcpStream`s.
//!
//! [`TcpStream`]: struct.TcpStream.html
//! [`connect`]: struct.TcpStream.html#method.connect
//! [`ConnectFuture`]: struct.ConnectFuture.html
//! [`TcpListener`]: struct.TcpListener.html
//! [incoming_method]: struct.TcpListener.html#method.incoming
//! [`Incoming`]: struct.Incoming.html
//!
//! # UDP
//!
//! The main struct for UDP is the [`UdpSocket`], which represents a UDP socket.
//! Reading and writing to it can be done using futures, which return the
//! [`RecvDgram`] and [`SendDgram`] structs respectively.
//!
//! For convience it's also possible to convert raw datagrams into higher-level
//! frames.
//!
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`RecvDgram`]: struct.RecvDgram.html
//! [`SendDgram`]: struct.SendDgram.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`framed`]: struct.UdpSocket.html#method.framed
mod tcp;
mod udp;
pub use self::tcp::{TcpStream, ConnectFuture};
pub use self::tcp::{TcpListener, Incoming};
pub use self::udp::{UdpSocket, UdpFramed, SendDgram, RecvDgram};
-213
View File
@@ -1,213 +0,0 @@
use net::tcp::Incoming;
use net::tcp::TcpStream;
use std::fmt;
use std::io;
use std::net::{self, SocketAddr};
use futures::{Poll, Async};
use mio;
use reactor::{Handle, PollEvented2};
/// An I/O object representing a TCP socket listening for incoming connections.
///
/// This object can be converted into a stream of incoming connections for
/// various forms of processing.
pub struct TcpListener {
io: PollEvented2<mio::net::TcpListener>,
}
impl TcpListener {
/// Create a new TCP listener associated with this event loop.
///
/// The TCP listener will bind to the provided `addr` address, if available.
/// If the result is `Ok`, the socket has successfully bound.
pub fn bind(addr: &SocketAddr) -> io::Result<TcpListener> {
let l = mio::net::TcpListener::bind(addr)?;
Ok(TcpListener::new(l))
}
#[deprecated(since = "0.1.2", note = "use poll_accept instead")]
#[doc(hidden)]
pub fn accept(&mut self) -> io::Result<(TcpStream, SocketAddr)> {
match self.poll_accept()? {
Async::Ready(ret) => Ok(ret),
Async::NotReady => Err(io::ErrorKind::WouldBlock.into()),
}
}
/// Attempt to accept a connection and create a new connected `TcpStream` if
/// successful.
///
/// Note that typically for simple usage it's easier to treat incoming
/// connections as a `Stream` of `TcpStream`s with the `incoming` method
/// below.
///
/// # Return
///
/// On success, returns `Ok(Async::Ready((socket, addr)))`.
///
/// If the listener is not ready to accept, the method returns
/// `Ok(Async::NotReady)` and arranges for the current task to receive a
/// notification when the listener becomes ready to accept.
///
/// # Panics
///
/// This function will panic if called from outside of a task context.
pub fn poll_accept(&mut self) -> Poll<(TcpStream, SocketAddr), io::Error> {
let (io, addr) = try_ready!(self.poll_accept_std());
let io = mio::net::TcpStream::from_stream(io)?;
let io = TcpStream::new(io);
Ok((io, addr).into())
}
#[deprecated(since = "0.1.2", note = "use poll_accept_std instead")]
#[doc(hidden)]
pub fn accept_std(&mut self) -> io::Result<(net::TcpStream, SocketAddr)> {
match self.poll_accept_std()? {
Async::Ready(ret) => Ok(ret),
Async::NotReady => Err(io::ErrorKind::WouldBlock.into()),
}
}
/// Attempt to accept a connection and create a new connected `TcpStream` if
/// successful.
///
/// This function is the asme as `accept` above except that it returns a
/// `std::net::TcpStream` instead of a `tokio::net::TcpStream`. This in turn
/// can then allow for the TCP stream to be assoiated with a different
/// reactor than the one this `TcpListener` is associated with.
///
/// # Return
///
/// On success, returns `Ok(Async::Ready((socket, addr)))`.
///
/// If the listener is not ready to accept, the method returns
/// `Ok(Async::NotReady)` and arranges for the current task to receive a
/// notification when the listener becomes ready to accept.
///
/// # Panics
///
/// This function will panic if called from outside of a task context.
pub fn poll_accept_std(&mut self) -> Poll<(net::TcpStream, SocketAddr), io::Error> {
try_ready!(self.io.poll_read_ready(mio::Ready::readable()));
match self.io.get_ref().accept_std() {
Ok(pair) => Ok(pair.into()),
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
self.io.clear_read_ready(mio::Ready::readable())?;
Ok(Async::NotReady)
}
Err(e) => Err(e),
}
}
/// Create a new TCP listener from the standard library's TCP listener.
///
/// This method can be used when the `Handle::tcp_listen` method isn't
/// sufficient because perhaps some more configuration is needed in terms of
/// before the calls to `bind` and `listen`.
///
/// This API is typically paired with the `net2` crate and the `TcpBuilder`
/// type to build up and customize a listener before it's shipped off to the
/// backing event loop. This allows configuration of options like
/// `SO_REUSEPORT`, binding to multiple addresses, etc.
///
/// The `addr` argument here is one of the addresses that `listener` is
/// bound to and the listener will only be guaranteed to accept connections
/// of the same address type currently.
///
/// Finally, the `handle` argument is the event loop that this listener will
/// be bound to.
///
/// The platform specific behavior of this function looks like:
///
/// * On Unix, the socket is placed into nonblocking mode and connections
/// can be accepted as normal
///
/// * On Windows, the address is stored internally and all future accepts
/// will only be for the same IP version as `addr` specified. That is, if
/// `addr` is an IPv4 address then all sockets accepted will be IPv4 as
/// well (same for IPv6).
pub fn from_std(listener: net::TcpListener, handle: &Handle)
-> io::Result<TcpListener>
{
let io = mio::net::TcpListener::from_std(listener)?;
let io = PollEvented2::new_with_handle(io, handle)?;
Ok(TcpListener { io })
}
fn new(listener: mio::net::TcpListener) -> TcpListener {
let io = PollEvented2::new(listener);
TcpListener { io }
}
/// Returns the local address that this listener is bound to.
///
/// This can be useful, for example, when binding to port 0 to figure out
/// which port was actually bound.
pub fn local_addr(&self) -> io::Result<SocketAddr> {
self.io.get_ref().local_addr()
}
/// Consumes this listener, returning a stream of the sockets this listener
/// accepts.
///
/// This method returns an implementation of the `Stream` trait which
/// resolves to the sockets the are accepted on this listener.
pub fn incoming(self) -> Incoming {
Incoming::new(self)
}
/// Gets the value of the `IP_TTL` option for this socket.
///
/// For more information about this option, see [`set_ttl`].
///
/// [`set_ttl`]: #method.set_ttl
pub fn ttl(&self) -> io::Result<u32> {
self.io.get_ref().ttl()
}
/// Sets the value for the `IP_TTL` option on this socket.
///
/// This value sets the time-to-live field that is used in every packet sent
/// from this socket.
pub fn set_ttl(&self, ttl: u32) -> io::Result<()> {
self.io.get_ref().set_ttl(ttl)
}
}
impl fmt::Debug for TcpListener {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
self.io.get_ref().fmt(f)
}
}
#[cfg(all(unix, not(target_os = "fuchsia")))]
mod sys {
use std::os::unix::prelude::*;
use super::TcpListener;
impl AsRawFd for TcpListener {
fn as_raw_fd(&self) -> RawFd {
self.io.get_ref().as_raw_fd()
}
}
}
#[cfg(windows)]
mod sys {
// TODO: let's land these upstream with mio and then we can add them here.
//
// use std::os::windows::prelude::*;
// use super::{TcpListener;
//
// impl AsRawHandle for TcpListener {
// fn as_raw_handle(&self) -> RawHandle {
// self.listener.io().as_raw_handle()
// }
// }
}
-8
View File
@@ -1,8 +0,0 @@
mod incoming;
mod listener;
mod stream;
pub use self::incoming::Incoming;
pub use self::listener::TcpListener;
pub use self::stream::TcpStream;
pub use self::stream::ConnectFuture;
-565
View File
@@ -1,565 +0,0 @@
use std::fmt;
use std::io::{self, Read, Write};
use std::mem;
use std::net::{self, SocketAddr, Shutdown};
use std::time::Duration;
use bytes::{Buf, BufMut};
use futures::{Future, Poll, Async};
use iovec::IoVec;
use mio;
use tokio_io::{AsyncRead, AsyncWrite};
use reactor::{Handle, PollEvented2};
/// An I/O object representing a TCP stream connected to a remote endpoint.
///
/// A TCP stream can either be created by connecting to an endpoint, via the
/// [`connect`] method, or by [accepting] a connection from a [listener].
///
/// [`connect`]: struct.TcpStream.html#method.connect
/// [accepting]: struct.TcpListener.html#method.accept
/// [listener]: struct.TcpListener.html
pub struct TcpStream {
io: PollEvented2<mio::net::TcpStream>,
}
/// Future returned by `TcpStream::connect` which will resolve to a `TcpStream`
/// when the stream is connected.
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
pub struct ConnectFuture {
inner: ConnectFutureState,
}
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
enum ConnectFutureState {
Waiting(TcpStream),
Error(io::Error),
Empty,
}
impl TcpStream {
/// Create a new TCP stream connected to the specified address.
///
/// This function will create a new TCP socket and attempt to connect it to
/// the `addr` provided. The returned future will be resolved once the
/// stream has successfully connected, or it wil return an error if one
/// occurs.
pub fn connect(addr: &SocketAddr) -> ConnectFuture {
use self::ConnectFutureState::*;
let inner = match mio::net::TcpStream::connect(addr) {
Ok(tcp) => Waiting(TcpStream::new(tcp)),
Err(e) => Error(e),
};
ConnectFuture { inner }
}
pub(crate) fn new(connected: mio::net::TcpStream) -> TcpStream {
let io = PollEvented2::new(connected);
TcpStream { io }
}
/// Create a new `TcpStream` from a `net::TcpStream`.
///
/// This function will convert a TCP stream created by the standard library
/// to a TCP stream ready to be used with the provided event loop handle.
/// The stream returned is associated with the event loop and ready to
/// perform I/O.
pub fn from_std(stream: net::TcpStream, handle: &Handle)
-> io::Result<TcpStream>
{
let io = mio::net::TcpStream::from_stream(stream)?;
let io = PollEvented2::new_with_handle(io, handle)?;
Ok(TcpStream { io })
}
/// Creates a new `TcpStream` from the pending socket inside the given
/// `std::net::TcpStream`, connecting it to the address specified.
///
/// This constructor allows configuring the socket before it's actually
/// connected, and this function will transfer ownership to the returned
/// `TcpStream` if successful. An unconnected `TcpStream` can be created
/// with the `net2::TcpBuilder` type (and also configured via that route).
///
/// The platform specific behavior of this function looks like:
///
/// * On Unix, the socket is placed into nonblocking mode and then a
/// `connect` call is issued.
///
/// * On Windows, the address is stored internally and the connect operation
/// is issued when the returned `TcpStream` is registered with an event
/// loop. Note that on Windows you must `bind` a socket before it can be
/// connected, so if a custom `TcpBuilder` is used it should be bound
/// (perhaps to `INADDR_ANY`) before this method is called.
pub fn connect_std(stream: net::TcpStream,
addr: &SocketAddr,
handle: &Handle)
-> ConnectFuture
{
use self::ConnectFutureState::*;
let io = mio::net::TcpStream::connect_stream(stream, addr)
.and_then(|io| PollEvented2::new_with_handle(io, handle));
let inner = match io {
Ok(io) => Waiting(TcpStream { io }),
Err(e) => Error(e),
};
ConnectFuture { inner: inner }
}
/// Check the TCP stream's read readiness state.
///
/// The mask argument allows specifying what readiness to notify on. This
/// can be any value, including platform specific readiness, **except**
/// `writable`. HUP is always implicitly included on platforms that support
/// it.
///
/// If the resource is not ready for a read then `Async::NotReady` is
/// returned and the current task is notified once a new event is received.
///
/// The stream will remain in a read-ready state until calls to `poll_read`
/// return `NotReady`.
///
/// # Panics
///
/// This function panics if:
///
/// * `ready` includes writable.
/// * called from outside of a task context.
pub fn poll_read_ready(&self, mask: mio::Ready) -> Poll<mio::Ready, io::Error> {
self.io.poll_read_ready(mask)
}
/// Check the TCP stream's write readiness state.
///
/// This always checks for writable readiness and also checks for HUP
/// readiness on platforms that support it.
///
/// If the resource is not ready for a write then `Async::NotReady` is
/// returned and the current task is notified once a new event is received.
///
/// The I/O resource will remain in a write-ready state until calls to
/// `poll_write` return `NotReady`.
///
/// # Panics
///
/// This function panics if:
///
/// * `ready` contains bits besides `writable` and `hup`.
/// * called from outside of a task context.
pub fn poll_write_ready(&self) -> Poll<mio::Ready, io::Error> {
self.io.poll_write_ready()
}
/// Returns the local address that this stream is bound to.
pub fn local_addr(&self) -> io::Result<SocketAddr> {
self.io.get_ref().local_addr()
}
/// Returns the remote address that this stream is connected to.
pub fn peer_addr(&self) -> io::Result<SocketAddr> {
self.io.get_ref().peer_addr()
}
#[deprecated(since = "0.1.2", note = "use poll_peek instead")]
#[doc(hidden)]
pub fn peek(&mut self, buf: &mut [u8]) -> io::Result<usize> {
match self.poll_peek(buf)? {
Async::Ready(n) => Ok(n),
Async::NotReady => Err(io::ErrorKind::WouldBlock.into()),
}
}
/// Receives data on the socket from the remote address to which it is
/// connected, without removing that data from the queue. On success,
/// returns the number of bytes peeked.
///
/// Successive calls return the same data. This is accomplished by passing
/// `MSG_PEEK` as a flag to the underlying recv system call.
///
/// # Return
///
/// On success, returns `Ok(Async::Ready(num_bytes_read))`.
///
/// If no data is available for reading, the method returns
/// `Ok(Async::NotReady)` and arranges for the current task to receive a
/// notification when the socket becomes readable or is closed.
///
/// # Panics
///
/// This function will panic if called from outside of a task context.
pub fn poll_peek(&mut self, buf: &mut [u8]) -> Poll<usize, io::Error> {
try_ready!(self.io.poll_read_ready(mio::Ready::readable()));
match self.io.get_ref().peek(buf) {
Ok(ret) => Ok(ret.into()),
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
self.io.clear_read_ready(mio::Ready::readable())?;
Ok(Async::NotReady)
}
Err(e) => Err(e),
}
}
/// Shuts down the read, write, or both halves of this connection.
///
/// This function will cause all pending and future I/O on the specified
/// portions to return immediately with an appropriate value (see the
/// documentation of `Shutdown`).
pub fn shutdown(&self, how: Shutdown) -> io::Result<()> {
self.io.get_ref().shutdown(how)
}
/// Gets the value of the `TCP_NODELAY` option on this socket.
///
/// For more information about this option, see [`set_nodelay`].
///
/// [`set_nodelay`]: #method.set_nodelay
pub fn nodelay(&self) -> io::Result<bool> {
self.io.get_ref().nodelay()
}
/// Sets the value of the `TCP_NODELAY` option on this socket.
///
/// If set, this option disables the Nagle algorithm. This means that
/// segments are always sent as soon as possible, even if there is only a
/// small amount of data. When not set, data is buffered until there is a
/// sufficient amount to send out, thereby avoiding the frequent sending of
/// small packets.
pub fn set_nodelay(&self, nodelay: bool) -> io::Result<()> {
self.io.get_ref().set_nodelay(nodelay)
}
/// Gets the value of the `SO_RCVBUF` option on this socket.
///
/// For more information about this option, see [`set_recv_buffer_size`].
///
/// [`set_recv_buffer_size`]: #tymethod.set_recv_buffer_size
pub fn recv_buffer_size(&self) -> io::Result<usize> {
self.io.get_ref().recv_buffer_size()
}
/// Sets the value of the `SO_RCVBUF` option on this socket.
///
/// Changes the size of the operating system's receive buffer associated
/// with the socket.
pub fn set_recv_buffer_size(&self, size: usize) -> io::Result<()> {
self.io.get_ref().set_recv_buffer_size(size)
}
/// Gets the value of the `SO_SNDBUF` option on this socket.
///
/// For more information about this option, see [`set_send_buffer`].
///
/// [`set_send_buffer`]: #tymethod.set_send_buffer
pub fn send_buffer_size(&self) -> io::Result<usize> {
self.io.get_ref().send_buffer_size()
}
/// Sets the value of the `SO_SNDBUF` option on this socket.
///
/// Changes the size of the operating system's send buffer associated with
/// the socket.
pub fn set_send_buffer_size(&self, size: usize) -> io::Result<()> {
self.io.get_ref().set_send_buffer_size(size)
}
/// Returns whether keepalive messages are enabled on this socket, and if so
/// the duration of time between them.
///
/// For more information about this option, see [`set_keepalive`].
///
/// [`set_keepalive`]: #tymethod.set_keepalive
pub fn keepalive(&self) -> io::Result<Option<Duration>> {
self.io.get_ref().keepalive()
}
/// Sets whether keepalive messages are enabled to be sent on this socket.
///
/// On Unix, this option will set the `SO_KEEPALIVE` as well as the
/// `TCP_KEEPALIVE` or `TCP_KEEPIDLE` option (depending on your platform).
/// On Windows, this will set the `SIO_KEEPALIVE_VALS` option.
///
/// If `None` is specified then keepalive messages are disabled, otherwise
/// the duration specified will be the time to remain idle before sending a
/// TCP keepalive probe.
///
/// Some platforms specify this value in seconds, so sub-second
/// specifications may be omitted.
pub fn set_keepalive(&self, keepalive: Option<Duration>) -> io::Result<()> {
self.io.get_ref().set_keepalive(keepalive)
}
/// Gets the value of the `IP_TTL` option for this socket.
///
/// For more information about this option, see [`set_ttl`].
///
/// [`set_ttl`]: #tymethod.set_ttl
pub fn ttl(&self) -> io::Result<u32> {
self.io.get_ref().ttl()
}
/// Sets the value for the `IP_TTL` option on this socket.
///
/// This value sets the time-to-live field that is used in every packet sent
/// from this socket.
pub fn set_ttl(&self, ttl: u32) -> io::Result<()> {
self.io.get_ref().set_ttl(ttl)
}
/// Reads the linger duration for this socket by getting the `SO_LINGER`
/// option.
///
/// For more information about this option, see [`set_linger`].
///
/// [`set_linger`]: #tymethod.set_linger
pub fn linger(&self) -> io::Result<Option<Duration>> {
self.io.get_ref().linger()
}
/// Sets the linger duration of this socket by setting the `SO_LINGER`
/// option.
///
/// This option controls the action taken when a stream has unsent messages
/// and the stream is closed. If `SO_LINGER` is set, the system
/// shall block the process until it can transmit the data or until the
/// time expires.
///
/// If `SO_LINGER` is not specified, and the stream is closed, the system
/// handles the call in a way that allows the process to continue as quickly
/// as possible.
pub fn set_linger(&self, dur: Option<Duration>) -> io::Result<()> {
self.io.get_ref().set_linger(dur)
}
}
// ===== impl Read / Write =====
impl Read for TcpStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.io.read(buf)
}
}
impl Write for TcpStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.io.write(buf)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl AsyncRead for TcpStream {
unsafe fn prepare_uninitialized_buffer(&self, _: &mut [u8]) -> bool {
false
}
fn read_buf<B: BufMut>(&mut self, buf: &mut B) -> Poll<usize, io::Error> {
<&TcpStream>::read_buf(&mut &*self, buf)
}
}
impl AsyncWrite for TcpStream {
fn shutdown(&mut self) -> Poll<(), io::Error> {
<&TcpStream>::shutdown(&mut &*self)
}
fn write_buf<B: Buf>(&mut self, buf: &mut B) -> Poll<usize, io::Error> {
<&TcpStream>::write_buf(&mut &*self, buf)
}
}
// ===== impl Read / Write for &'a =====
impl<'a> Read for &'a TcpStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
(&self.io).read(buf)
}
}
impl<'a> Write for &'a TcpStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
(&self.io).write(buf)
}
fn flush(&mut self) -> io::Result<()> {
(&self.io).flush()
}
}
impl<'a> AsyncRead for &'a TcpStream {
unsafe fn prepare_uninitialized_buffer(&self, _: &mut [u8]) -> bool {
false
}
fn read_buf<B: BufMut>(&mut self, buf: &mut B) -> Poll<usize, io::Error> {
if let Async::NotReady = self.io.poll_read_ready(mio::Ready::readable())? {
return Ok(Async::NotReady)
}
let r = unsafe {
// The `IoVec` type can't have a 0-length size, so we create a bunch
// of dummy versions on the stack with 1 length which we'll quickly
// overwrite.
let b1: &mut [u8] = &mut [0];
let b2: &mut [u8] = &mut [0];
let b3: &mut [u8] = &mut [0];
let b4: &mut [u8] = &mut [0];
let b5: &mut [u8] = &mut [0];
let b6: &mut [u8] = &mut [0];
let b7: &mut [u8] = &mut [0];
let b8: &mut [u8] = &mut [0];
let b9: &mut [u8] = &mut [0];
let b10: &mut [u8] = &mut [0];
let b11: &mut [u8] = &mut [0];
let b12: &mut [u8] = &mut [0];
let b13: &mut [u8] = &mut [0];
let b14: &mut [u8] = &mut [0];
let b15: &mut [u8] = &mut [0];
let b16: &mut [u8] = &mut [0];
let mut bufs: [&mut IoVec; 16] = [
b1.into(), b2.into(), b3.into(), b4.into(),
b5.into(), b6.into(), b7.into(), b8.into(),
b9.into(), b10.into(), b11.into(), b12.into(),
b13.into(), b14.into(), b15.into(), b16.into(),
];
let n = buf.bytes_vec_mut(&mut bufs);
self.io.get_ref().read_bufs(&mut bufs[..n])
};
match r {
Ok(n) => {
unsafe { buf.advance_mut(n); }
Ok(Async::Ready(n))
}
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
self.io.clear_read_ready(mio::Ready::readable())?;
Ok(Async::NotReady)
}
Err(e) => Err(e),
}
}
}
impl<'a> AsyncWrite for &'a TcpStream {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
fn write_buf<B: Buf>(&mut self, buf: &mut B) -> Poll<usize, io::Error> {
if let Async::NotReady = self.io.poll_write_ready()? {
return Ok(Async::NotReady)
}
let r = {
// The `IoVec` type can't have a zero-length size, so create a dummy
// version from a 1-length slice which we'll overwrite with the
// `bytes_vec` method.
static DUMMY: &[u8] = &[0];
let iovec = <&IoVec>::from(DUMMY);
let mut bufs = [iovec; 64];
let n = buf.bytes_vec(&mut bufs);
self.io.get_ref().write_bufs(&bufs[..n])
};
match r {
Ok(n) => {
buf.advance(n);
Ok(Async::Ready(n))
}
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
self.io.clear_write_ready()?;
Ok(Async::NotReady)
}
Err(e) => Err(e),
}
}
}
impl fmt::Debug for TcpStream {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
self.io.get_ref().fmt(f)
}
}
impl Future for ConnectFuture {
type Item = TcpStream;
type Error = io::Error;
fn poll(&mut self) -> Poll<TcpStream, io::Error> {
self.inner.poll()
}
}
impl Future for ConnectFutureState {
type Item = TcpStream;
type Error = io::Error;
fn poll(&mut self) -> Poll<TcpStream, io::Error> {
{
let stream = match *self {
ConnectFutureState::Waiting(ref mut s) => s,
ConnectFutureState::Error(_) => {
let e = match mem::replace(self, ConnectFutureState::Empty) {
ConnectFutureState::Error(e) => e,
_ => panic!(),
};
return Err(e)
}
ConnectFutureState::Empty => panic!("can't poll TCP stream twice"),
};
// Once we've connected, wait for the stream to be writable as
// that's when the actual connection has been initiated. Once we're
// writable we check for `take_socket_error` to see if the connect
// actually hit an error or not.
//
// If all that succeeded then we ship everything on up.
if let Async::NotReady = stream.io.poll_write_ready()? {
return Ok(Async::NotReady)
}
if let Some(e) = try!(stream.io.get_ref().take_error()) {
return Err(e)
}
}
match mem::replace(self, ConnectFutureState::Empty) {
ConnectFutureState::Waiting(stream) => Ok(Async::Ready(stream)),
_ => panic!(),
}
}
}
#[cfg(all(unix, not(target_os = "fuchsia")))]
mod sys {
use std::os::unix::prelude::*;
use super::TcpStream;
impl AsRawFd for TcpStream {
fn as_raw_fd(&self) -> RawFd {
self.io.get_ref().as_raw_fd()
}
}
}
#[cfg(windows)]
mod sys {
// TODO: let's land these upstream with mio and then we can add them here.
//
// use std::os::windows::prelude::*;
// use super::TcpStream;
//
// impl AsRawHandle for TcpStream {
// fn as_raw_handle(&self) -> RawHandle {
// self.io.get_ref().as_raw_handle()
// }
// }
}
-9
View File
@@ -1,9 +0,0 @@
mod frame;
mod socket;
mod send_dgram;
mod recv_dgram;
pub use self::frame::UdpFramed;
pub use self::socket::UdpSocket;
pub use self::send_dgram::SendDgram;
pub use self::recv_dgram::RecvDgram;
-52
View File
@@ -1,52 +0,0 @@
use net::udp::socket::UdpSocket;
use std::io;
use std::net::SocketAddr;
use futures::{Async, Future, Poll};
/// A future used to receive a datagram from a UDP socket.
///
/// This is created by the `UdpSocket::recv_dgram` method.
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
pub struct RecvDgram<T> {
/// None means future was completed
state: Option<RecvDgramInner<T>>
}
/// A struct is used to represent the full info of RecvDgram.
#[derive(Debug)]
struct RecvDgramInner<T> {
/// Rx socket
socket: UdpSocket,
/// The received data will be put in the buffer
buffer: T
}
impl<T> RecvDgram<T> {
/// Create a new future to receive UDP Datagram
pub(crate) fn new(socket: UdpSocket, buffer: T) -> RecvDgram<T> {
let inner = RecvDgramInner { socket: socket, buffer: buffer };
RecvDgram { state: Some(inner) }
}
}
impl<T> Future for RecvDgram<T>
where T: AsMut<[u8]>,
{
type Item = (UdpSocket, T, usize, SocketAddr);
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, io::Error> {
let (n, addr) = {
let ref mut inner =
self.state.as_mut().expect("RecvDgram polled after completion");
try_ready!(inner.socket.poll_recv_from(inner.buffer.as_mut()))
};
let inner = self.state.take().unwrap();
Ok(Async::Ready((inner.socket, inner.buffer, n, addr)))
}
}
-61
View File
@@ -1,61 +0,0 @@
use net::udp::socket::UdpSocket;
use std::io;
use std::net::SocketAddr;
use futures::{Async, Future, Poll};
/// A future used to write the entire contents of some data to a UDP socket.
///
/// This is created by the `UdpSocket::send_dgram` method.
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
pub struct SendDgram<T> {
/// None means future was completed
state: Option<SendDgramInner<T>>
}
/// A struct is used to represent the full info of SendDgram.
#[derive(Debug)]
struct SendDgramInner<T> {
/// Tx socket
socket: UdpSocket,
/// The whole buffer will be sent
buffer: T,
/// Destination addr
addr: SocketAddr,
}
impl<T> SendDgram<T> {
/// Create a new future to send UDP Datagram
pub(crate) fn new(socket: UdpSocket, buffer: T, addr: SocketAddr) -> SendDgram<T> {
let inner = SendDgramInner { socket: socket, buffer: buffer, addr: addr };
SendDgram { state: Some(inner) }
}
}
fn incomplete_write(reason: &str) -> io::Error {
io::Error::new(io::ErrorKind::Other, reason)
}
impl<T> Future for SendDgram<T>
where T: AsRef<[u8]>,
{
type Item = (UdpSocket, T);
type Error = io::Error;
fn poll(&mut self) -> Poll<(UdpSocket, T), io::Error> {
{
let ref mut inner =
self.state.as_mut().expect("SendDgram polled after completion");
let n = try_ready!(inner.socket.poll_send_to(inner.buffer.as_ref(), &inner.addr));
if n != inner.buffer.as_ref().len() {
return Err(incomplete_write("failed to send entire message \
in datagram"))
}
}
let inner = self.state.take().unwrap();
Ok(Async::Ready((inner.socket, inner.buffer)))
}
}
-549
View File
@@ -1,549 +0,0 @@
//! Readiness tracking streams, backing I/O objects.
//!
//! This module contains the core type which is used to back all I/O on object
//! in `tokio-core`. The `PollEvented` type is the implementation detail of
//! all I/O. Each `PollEvented` manages registration with a reactor,
//! acquisition of a token, and tracking of the readiness state on the
//! underlying I/O primitive.
#![allow(deprecated, warnings)]
use std::fmt;
use std::io::{self, Read, Write};
use std::sync::Mutex;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::{task, Async, Poll};
use mio::event::Evented;
use mio::Ready;
use tokio_io::{AsyncRead, AsyncWrite};
use reactor::{Handle, Registration};
#[deprecated(since = "0.1.2", note = "PollEvented2 instead")]
#[doc(hidden)]
pub struct PollEvented<E> {
io: E,
inner: Inner,
handle: Handle,
}
struct Inner {
registration: Mutex<Registration>,
/// Currently visible read readiness
read_readiness: AtomicUsize,
/// Currently visible write readiness
write_readiness: AtomicUsize,
}
impl<E: fmt::Debug> fmt::Debug for PollEvented<E> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("PollEvented")
.field("io", &self.io)
.finish()
}
}
impl<E> PollEvented<E> {
/// Creates a new readiness stream associated with the provided
/// `loop_handle` and for the given `source`.
pub fn new(io: E, handle: &Handle) -> io::Result<PollEvented<E>>
where E: Evented,
{
let registration = Registration::new();
registration.register(&io)?;
Ok(PollEvented {
io: io,
inner: Inner {
registration: Mutex::new(registration),
read_readiness: AtomicUsize::new(0),
write_readiness: AtomicUsize::new(0),
},
handle: handle.clone(),
})
}
/// Tests to see if this source is ready to be read from or not.
///
/// If this stream is not ready for a read then `Async::NotReady` will be
/// returned and the current task will be scheduled to receive a
/// notification when the stream is readable again. In other words, this
/// method is only safe to call from within the context of a future's task,
/// typically done in a `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::readable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_read(&mut self) -> Async<()> {
if self.poll_read2().is_ready() {
return ().into();
}
Async::NotReady
}
fn poll_read2(&self) -> Async<Ready> {
let r = self.inner.registration.lock().unwrap();
// Load the cached readiness
match self.inner.read_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_read_ready().unwrap() {
n |= ready2usize(ready);
self.inner.read_readiness.store(n, Relaxed);
}
return usize2ready(n).into();
}
}
let ready = match r.poll_read_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.read_readiness.store(ready2usize(ready), Relaxed);
ready.into()
}
/// Tests to see if this source is ready to be written to or not.
///
/// If this stream is not ready for a write then `Async::NotReady` will be returned
/// and the current task will be scheduled to receive a notification when
/// the stream is writable again. In other words, this method is only safe
/// to call from within the context of a future's task, typically done in a
/// `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::writable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_write(&mut self) -> Async<()> {
let r = self.inner.registration.lock().unwrap();
match self.inner.write_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_write_ready().unwrap() {
n |= ready2usize(ready);
self.inner.write_readiness.store(n, Relaxed);
}
return ().into();
}
}
let ready = match r.poll_write_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.write_readiness.store(ready2usize(ready), Relaxed);
().into()
}
/// Test to see whether this source fulfills any condition listed in `mask`
/// provided.
///
/// The `mask` given here is a mio `Ready` set of possible events. This can
/// contain any events like read/write but also platform-specific events
/// such as hup and error. The `mask` indicates events that are interested
/// in being ready.
///
/// If any event in `mask` is ready then it is returned through
/// `Async::Ready`. The `Ready` set returned is guaranteed to not be empty
/// and contains all events that are currently ready in the `mask` provided.
///
/// If no events are ready in the `mask` provided then the current task is
/// scheduled to receive a notification when any of them become ready. If
/// the `writable` event is contained within `mask` then this
/// `PollEvented`'s `write` task will be blocked and otherwise the `read`
/// task will be blocked. This is generally only relevant if you're working
/// with this `PollEvented` object on multiple tasks.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_ready(&mut self, mask: Ready) -> Async<Ready> {
let mut ret = Ready::empty();
if mask.is_empty() {
return ret.into();
}
if mask.is_writable() {
if self.poll_write().is_ready() {
ret = Ready::writable();
}
}
let mask = mask - Ready::writable();
if !mask.is_empty() {
if let Async::Ready(v) = self.poll_read2() {
ret |= v & mask;
}
}
if ret.is_empty() {
if mask.is_writable() {
let _ = self.need_write();
}
if mask.is_readable() {
let _ = self.need_read();
}
Async::NotReady
} else {
ret.into()
}
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer readable, but it needs to be.
///
/// This function, like `poll_read`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// readable, typically because a "would block" error was seen.
///
/// *All* readiness bits associated with this stream except the writable bit
/// will be reset when this method is called. The current task is then
/// scheduled to receive a notification whenever anything changes other than
/// the writable bit. Note that this typically just means the readable bit
/// is used here, but if you're using a custom I/O object for events like
/// hup/error this may also be relevant.
///
/// Note that it is also only valid to call this method if `poll_read`
/// previously indicated that the object is readable. That is, this function
/// must always be paired with calls to `poll_read` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_read(&mut self) -> io::Result<()> {
self.inner.read_readiness.store(0, Relaxed);
if self.poll_read().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer writable, but it needs to be.
///
/// This function, like `poll_write`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// writable, typically because a "would block" error was seen.
///
/// The flag indicating that this stream is writable is unset and the
/// current task is scheduled to receive a notification when the stream is
/// then again writable.
///
/// Note that it is also only valid to call this method if `poll_write`
/// previously indicated that the object is writable. That is, this function
/// must always be paired with calls to `poll_write` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_write(&mut self) -> io::Result<()> {
self.inner.write_readiness.store(0, Relaxed);
if self.poll_write().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Returns a reference to the event loop handle that this readiness stream
/// is associated with.
pub fn handle(&self) -> &Handle {
&self.handle
}
/// Returns a shared reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_ref(&self) -> &E {
&self.io
}
/// Returns a mutable reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_mut(&mut self) -> &mut E {
&mut self.io
}
/// Consumes the `PollEvented` and returns the underlying I/O object
pub fn into_inner(self) -> E {
self.io
}
/// Deregisters this source of events from the reactor core specified.
///
/// This method can optionally be called to unregister the underlying I/O
/// object with the event loop that the `handle` provided points to.
/// Typically this method is not required as this automatically happens when
/// `E` is dropped, but for some use cases the `E` object doesn't represent
/// an owned reference, so dropping it won't automatically unregister with
/// the event loop.
///
/// This consumes `self` as it will no longer provide events after the
/// method is called, and will likely return an error if this `PollEvented`
/// was created on a separate event loop from the `handle` specified.
pub fn deregister(&self) -> io::Result<()>
where E: Evented,
{
self.inner.registration.lock().unwrap()
.deregister(&self.io)
}
}
impl<E: Read> Read for PollEvented<E> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_read() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().read(buf);
if is_wouldblock(&r) {
self.need_read()?;
}
return r
}
}
impl<E: Write> Write for PollEvented<E> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().write(buf);
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
fn flush(&mut self) -> io::Result<()> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().flush();
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
}
impl<E: Read> AsyncRead for PollEvented<E> {
}
impl<E: Write> AsyncWrite for PollEvented<E> {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
fn is_wouldblock<T>(r: &io::Result<T>) -> bool {
match *r {
Ok(_) => false,
Err(ref e) => e.kind() == io::ErrorKind::WouldBlock,
}
}
const READ: usize = 1 << 0;
const WRITE: usize = 1 << 1;
fn ready2usize(ready: Ready) -> usize {
let mut bits = 0;
if ready.is_readable() {
bits |= READ;
}
if ready.is_writable() {
bits |= WRITE;
}
bits | platform::ready2usize(ready)
}
fn usize2ready(bits: usize) -> Ready {
let mut ready = Ready::empty();
if bits & READ != 0 {
ready.insert(Ready::readable());
}
if bits & WRITE != 0 {
ready.insert(Ready::writable());
}
ready | platform::usize2ready(bits)
}
#[cfg(all(unix, not(target_os = "fuchsia")))]
mod platform {
use mio::Ready;
use mio::unix::UnixReady;
#[cfg(target_os = "dragonfly")]
pub fn all() -> Ready {
hup() | UnixReady::aio()
}
#[cfg(target_os = "freebsd")]
pub fn all() -> Ready {
hup() | UnixReady::aio() | UnixReady::lio()
}
const HUP: usize = 1 << 2;
const ERROR: usize = 1 << 3;
const AIO: usize = 1 << 4;
const LIO: usize = 1 << 5;
#[cfg(any(target_os = "dragonfly", target_os = "freebsd"))]
fn is_aio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_aio()
}
#[cfg(not(any(target_os = "dragonfly", target_os = "freebsd")))]
fn is_aio(_ready: &Ready) -> bool {
false
}
#[cfg(target_os = "freebsd")]
fn is_lio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_lio()
}
#[cfg(not(target_os = "freebsd"))]
fn is_lio(_ready: &Ready) -> bool {
false
}
pub fn ready2usize(ready: Ready) -> usize {
let ready = UnixReady::from(ready);
let mut bits = 0;
if is_aio(&ready) {
bits |= AIO;
}
if is_lio(&ready) {
bits |= LIO;
}
if ready.is_error() {
bits |= ERROR;
}
if ready.is_hup() {
bits |= HUP;
}
bits
}
#[cfg(any(target_os = "dragonfly", target_os = "freebsd", target_os = "ios",
target_os = "macos"))]
fn usize2ready_aio(ready: &mut UnixReady) {
ready.insert(UnixReady::aio());
}
#[cfg(not(any(target_os = "dragonfly",
target_os = "freebsd", target_os = "ios", target_os = "macos")))]
fn usize2ready_aio(_ready: &mut UnixReady) {
// aio not available here → empty
}
#[cfg(target_os = "freebsd")]
fn usize2ready_lio(ready: &mut UnixReady) {
ready.insert(UnixReady::lio());
}
#[cfg(not(target_os = "freebsd"))]
fn usize2ready_lio(_ready: &mut UnixReady) {
// lio not available here → empty
}
pub fn usize2ready(bits: usize) -> Ready {
let mut ready = UnixReady::from(Ready::empty());
if bits & AIO != 0 {
usize2ready_aio(&mut ready);
}
if bits & LIO != 0 {
usize2ready_lio(&mut ready);
}
if bits & HUP != 0 {
ready.insert(UnixReady::hup());
}
if bits & ERROR != 0 {
ready.insert(UnixReady::error());
}
ready.into()
}
}
#[cfg(any(windows, target_os = "fuchsia"))]
mod platform {
use mio::Ready;
pub fn all() -> Ready {
// No platform-specific Readinesses for Windows
Ready::empty()
}
pub fn hup() -> Ready {
Ready::empty()
}
pub fn ready2usize(_r: Ready) -> usize {
0
}
pub fn usize2ready(_r: usize) -> Ready {
Ready::empty()
}
}
-444
View File
@@ -1,444 +0,0 @@
//! A batteries included runtime for applications using Tokio.
//!
//! Applications using Tokio require some runtime support in order to work:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
//!
//! [`Runtime`] bundles all of these various runtime components into a single
//! handle that can be started and shutdown together, eliminating the necessary
//! boilerplate to run a Tokio application.
//!
//! Most applications wont need to use [`Runtime`] directly. Instead, they will
//! use the [`run`] function, which uses [`Runtime`] under the hood.
//!
//! Creating a [`Runtime`] does the following:
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
//! "seed" the application, blocking the thread until the runtime becomes
//! [idle].
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! tokio::run(server);
//! # }
//! # pub fn main() {}
//! ```
//!
//! In this function, the `run` function blocks until the runtime becomes idle.
//! See [`shutdown_on_idle`][idle] for more shutdown details.
//!
//! From within the context of the runtime, additional tasks are spawned using
//! the [`tokio::spawn`] function. Futures spawned using this function will be
//! executed on the same thread pool used by the [`Runtime`].
//!
//! A [`Runtime`] instance can also be used directly.
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::runtime::Runtime;
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! // Create the runtime
//! let mut rt = Runtime::new().unwrap();
//!
//! // Spawn the server task
//! rt.spawn(server);
//!
//! // Wait until the runtime becomes idle and shut it down.
//! rt.shutdown_on_idle()
//! .wait().unwrap();
//! # }
//! # pub fn main() {}
//! ```
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [`Runtime`]: struct.Runtime.html
//! [`ThreadPool`]: ../executor/thread_pool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
use reactor::{Reactor, Handle, Background};
use tokio_threadpool::{self as threadpool, ThreadPool, Sender};
use futures::Poll;
use futures::future::{self, Future};
use std::{fmt, io};
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
inner: Sender,
}
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
inner: Box<Future<Item = (), Error = ()> + Send>,
}
#[derive(Debug)]
struct Inner {
/// Reactor running on a background thread.
reactor: Background,
/// Task execution pool.
pool: ThreadPool,
}
// ===== impl Runtime =====
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// This function is used to bootstrap the execution of a Tokio application. It
/// does the following:
///
/// * Start the Tokio runtime using a default configuration.
/// * Spawn the given future onto the thread pool.
/// * Block the current thread until the runtime shuts down.
///
/// Note that the function will not return immediately once `future` has
/// completed. Instead it waits for the entire runtime to become idle.
///
/// See the [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if called from the context of an executor.
///
/// [mod]: ../index.html
pub fn run<F>(future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
let mut runtime = Runtime::new().unwrap();
runtime.spawn(future);
runtime.shutdown_on_idle().wait().unwrap();
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
// Spawn a reactor on a background thread.
let reactor = Reactor::new()?.background()?;
// Get a handle to the reactor.
let handle = reactor.handle().clone();
let pool = threadpool::Builder::new()
.around_worker(move |w, enter| {
::tokio_reactor::with_default(&handle, enter, |_| {
w.run();
});
})
.build();
Ok(Runtime {
inner: Some(Inner {
reactor,
pool,
}),
})
}
/// Return a reference to the reactor handle for this runtime instance.
pub fn handle(&self) -> &Handle {
self.inner().reactor.handle()
}
/// Return a handle to the runtime's executor.
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
}
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner_mut().pool.sender().spawn(future).unwrap();
self
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function can be used to perform a graceful shutdown of the runtime.
///
/// The runtime enters an idle state once **all** of the following occur.
///
/// * The thread pool has no tasks to execute, i.e., all tasks that were
/// spawned have completed.
/// * The reactor is not managing any I/O resources.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_on_idle().and_then(|_| {
reactor.shutdown_on_idle()
})
});
Shutdown { inner }
}
/// Signals the runtime to shutdown immediately.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function will forcibly shutdown the runtime, causing any
/// in-progress work to become canceled. The shutdown steps are:
///
/// * Drain any scheduled work queues.
/// * Drop any futures that have not yet completed.
/// * Drop the reactor.
///
/// Once the reactor has dropped, any outstanding I/O resources bound to
/// that reactor will no longer function. Calling any method on them will
/// result in an error.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_now().and_then(|_| {
reactor.shutdown_now()
})
});
Shutdown { inner }
}
fn inner(&self) -> &Inner {
self.inner.as_ref().unwrap()
}
fn inner_mut(&mut self) -> &mut Inner {
self.inner.as_mut().unwrap()
}
}
// ===== impl TaskExecutor =====
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
}
// ===== impl Shutdown =====
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
-63
View File
@@ -1,63 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::net::TcpStream;
use std::thread;
use std::io::{Read, Write, BufReader, BufWriter};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::copy;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
const N: usize = 1024;
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let t2 = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let mut b = vec![0; msg.len() * N];
t!(s.read_exact(&mut b));
b
});
let mut expected = Vec::<u8>::new();
for _i in 0..N {
expected.extend(msg.as_bytes());
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
}
(expected, t2)
});
let clients = srv.incoming().take(2).collect();
let copied = clients.and_then(|clients| {
let mut clients = clients.into_iter();
let a = BufReader::new(clients.next().unwrap());
let b = BufWriter::new(clients.next().unwrap());
copy(a, b)
});
let (amt, _, _) = t!(copied.wait());
let (expected, t2) = t.join().unwrap();
let actual = t2.join().unwrap();
assert!(expected == actual);
assert_eq!(amt, msg.len() as u64 * 1024);
}
-49
View File
@@ -1,49 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::net::TcpStream;
use std::thread;
use std::io::{Write, Read};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::read_to_end;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn chain_clients() {
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
let mut s1 = TcpStream::connect(&addr).unwrap();
s1.write_all(b"foo ").unwrap();
let mut s2 = TcpStream::connect(&addr).unwrap();
s2.write_all(b"bar ").unwrap();
let mut s3 = TcpStream::connect(&addr).unwrap();
s3.write_all(b"baz").unwrap();
});
let clients = srv.incoming().take(3);
let copied = clients.collect().and_then(|clients| {
let mut clients = clients.into_iter();
let a = clients.next().unwrap();
let b = clients.next().unwrap();
let c = clients.next().unwrap();
read_to_end(a.chain(b).chain(c), Vec::new())
});
let (_, data) = t!(copied.wait());
t.join().unwrap();
assert_eq!(data, b"foo bar baz");
}
-395
View File
@@ -1,395 +0,0 @@
extern crate tokio;
extern crate tokio_executor;
extern crate futures;
use tokio::executor::current_thread::{self, block_on_all, CurrentThread};
use std::any::Any;
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::prelude::*;
use futures::sync::oneshot;
#[test]
fn spawn_from_block_on_all() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let msg = current_thread::block_on_all(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Rc::new(Cell::new(0));
let cnt2 = cnt.clone();
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
assert_eq!(1, cnt2.get());
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
for _ in 0..ITER {
let cnt = cnt.clone();
current_thread.spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok::<(), ()>(())
}));
}
current_thread.run().unwrap();
assert_eq!(cnt.get(), ITER);
}
#[test]
fn does_not_set_global_executor_by_default() {
use tokio_executor::Executor;
block_on_all(lazy(|| {
tokio_executor::DefaultExecutor::current()
.spawn(Box::new(lazy(|| ok())))
.unwrap_err();
ok()
})).unwrap();
}
#[test]
fn spawn_from_block_on_future() {
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
current_thread::spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
}));
Ok::<_, ()>(())
})).unwrap();
current_thread.run().unwrap();
assert_eq!(1, cnt.get());
}
struct Never(Rc<()>);
impl Future for Never {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(Async::NotReady)
}
}
#[test]
fn outstanding_tasks_are_dropped_when_executor_is_dropped() {
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.spawn(Never(rc.clone()));
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
// Using the global spawn fn
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
current_thread::spawn(Never(rc.clone()));
Ok::<_, ()>(())
})).unwrap();
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
}
#[test]
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
})).unwrap();
}
#[test]
#[should_panic]
fn run_in_future() {
block_on_all(lazy(|| {
current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
}));
ok()
})).unwrap();
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
struct Infini {
num: Rc<Cell<usize>>,
}
impl Future for Infini {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.num.set(1 + self.num.get());
task::current().notify();
Ok(Async::NotReady)
}
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.turn(None)
.unwrap();
assert_eq!(1, num.get());
}
#[test]
fn tasks_are_scheduled_fairly() {
let state = Rc::new(RefCell::new([0, 0]));
struct Spin {
state: Rc<RefCell<[i32; 2]>>,
idx: usize,
}
impl Future for Spin {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
let mut state = self.state.borrow_mut();
if self.idx == 0 {
let diff = state[0] - state[1];
assert!(diff.abs() <= 1);
if state[0] >= 50 {
return Ok(().into());
}
}
state[self.idx] += 1;
if state[self.idx] >= 100 {
return Ok(().into());
}
task::current().notify();
Ok(Async::NotReady)
}
}
block_on_all(lazy(|| {
current_thread::spawn(Spin {
state: state.clone(),
idx: 0,
});
current_thread::spawn(Spin {
state: state,
idx: 1,
});
ok()
})).unwrap();
}
#[test]
fn spawn_and_turn() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let mut current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
current_thread.spawn(lazy(move || {
Ok(())
}));
// Turn once...
current_thread.turn(None).unwrap();
current_thread.spawn(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok(())
}));
// This does not run the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(1, cnt.get());
// This runs the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(2, cnt.get());
}
#[test]
fn spawn_in_drop() {
let mut current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
current_thread.spawn({
struct OnDrop<F: FnOnce()>(Option<F>);
impl<F: FnOnce()> Drop for OnDrop<F> {
fn drop(&mut self) {
(self.0.take().unwrap())();
}
}
struct MyFuture {
_data: Box<Any>,
}
impl Future for MyFuture {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(().into())
}
}
MyFuture {
_data: Box::new(OnDrop(Some(move || {
current_thread::spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}));
}))),
}
});
current_thread.block_on(rx).unwrap();
current_thread.run().unwrap();
}
#[test]
fn hammer_turn() {
use futures::sync::mpsc;
const ITER: usize = 100;
const N: usize = 100;
const THREADS: usize = 4;
for _ in 0..ITER {
let mut ths = vec![];
// Add some jitter
for _ in 0..THREADS {
let th = thread::spawn(|| {
let mut current_thread = CurrentThread::new();
let (tx, rx) = mpsc::unbounded();
current_thread.spawn({
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
rx.for_each(move |_| {
c.set(1 + c.get());
Ok(())
})
.map_err(|e| panic!("err={:?}", e))
.map(move |v| {
assert_eq!(N, cnt.get());
v
})
});
thread::spawn(move || {
for _ in 0..N {
tx.unbounded_send(()).unwrap();
thread::yield_now();
}
});
while !current_thread.is_idle() {
current_thread.turn(None).unwrap();
}
});
ths.push(th);
}
for th in ths {
th.join().unwrap();
}
}
}
fn ok() -> future::FutureResult<(), ()> {
future::ok(())
}
-42
View File
@@ -1,42 +0,0 @@
extern crate tokio;
extern crate futures;
use std::thread;
use std::net;
use futures::future;
use futures::prelude::*;
use futures::sync::oneshot;
use tokio::net::TcpListener;
use tokio::reactor::Reactor;
#[test]
fn tcp_doesnt_block() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
drop(core);
assert!(listener.incoming().wait().next().unwrap().is_err());
}
#[test]
fn drop_wakes() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
let (tx, rx) = oneshot::channel::<()>();
let t = thread::spawn(move || {
let incoming = listener.incoming();
let new_socket = incoming.into_future().map_err(|_| ());
let drop_tx = future::lazy(|| {
drop(tx);
future::ok(())
});
assert!(new_socket.join(drop_tx).wait().is_err());
});
drop(rx.wait());
drop(core);
t.join().unwrap();
}
-51
View File
@@ -1,51 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::thread;
use futures::Future;
use futures::stream::Stream;
use tokio::net::TcpListener;
use tokio_io::AsyncRead;
use tokio_io::io::copy;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = TcpStream::connect(&addr).unwrap();
for _i in 0..1024 {
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
let mut buf = [0; 1024];
assert_eq!(t!(s.read(&mut buf)), msg.len());
assert_eq!(&buf[..msg.len()], msg.as_bytes());
}
});
let clients = srv.incoming();
let client = clients.into_future().map(|e| e.0.unwrap()).map_err(|e| e.0);
let halves = client.map(|s| s.split());
let copied = halves.and_then(|(a, b)| copy(a, b));
let (amt, _, _) = t!(copied.wait());
t.join().unwrap();
assert_eq!(amt, msg.len() as u64 * 1024);
}
-120
View File
@@ -1,120 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn hammer() {
let _ = env_logger::init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().into_future()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(mine.join(theirs).wait());
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
}).collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
}
struct Rd(Arc<TcpStream>);
struct Wr(Arc<TcpStream>);
impl io::Read for Rd {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
<&TcpStream>::read(&mut &*self.0, dst)
}
}
impl tokio_io::AsyncRead for Rd {
}
impl io::Write for Wr {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
<&TcpStream>::write(&mut &*self.0, src)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl tokio_io::AsyncWrite for Wr {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
#[test]
fn hammer_split() {
use tokio_io::io;
const N: usize = 100;
let _ = env_logger::init();
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mut rt = Runtime::new().unwrap();
fn split(socket: TcpStream) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let rd = io::read(rd, vec![0; 1])
.map(|_| ())
.map_err(|e| panic!("read error = {:?}", e));
let wr = io::write_all(wr, b"1")
.map(|_| ())
.map_err(|e| panic!("write error = {:?}", e));
tokio::spawn(rd);
tokio::spawn(wr);
}
rt.spawn({
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(|socket| {
split(socket);
Ok(())
})
});
for _ in 0..N {
rt.spawn({
TcpStream::connect(&addr)
.map_err(|e| panic!("connect error = {:?}", e))
.map(|socket| split(socket))
});
}
rt.shutdown_on_idle().wait().unwrap();
}
-43
View File
@@ -1,43 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::net::TcpStream;
use std::thread;
use std::io::{Write, Read};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::read_to_end;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn limit() {
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
let mut s1 = TcpStream::connect(&addr).unwrap();
s1.write_all(b"foo bar baz").unwrap();
});
let clients = srv.incoming().take(1);
let copied = clients.collect().and_then(|clients| {
let mut clients = clients.into_iter();
let a = clients.next().unwrap();
read_to_end(a.take(4), Vec::new())
});
let (_, data) = t!(copied.wait());
t.join().unwrap();
assert_eq!(data, b"foo ");
}
-87
View File
@@ -1,87 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate futures_cpupool;
extern crate tokio;
extern crate tokio_io;
extern crate bytes;
use std::io;
use std::net::Shutdown;
use bytes::{BytesMut, BufMut};
use futures::{Future, Stream, Sink};
use futures::future::Executor;
use futures_cpupool::CpuPool;
use tokio::net::{TcpListener, TcpStream};
use tokio_io::codec::{Encoder, Decoder};
use tokio_io::io::{write_all, read};
use tokio_io::AsyncRead;
pub struct LineCodec;
impl Decoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<BytesMut>, io::Error> {
match buf.iter().position(|&b| b == b'\n') {
Some(i) => Ok(Some(buf.split_to(i + 1).into())),
None => Ok(None),
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() == 0 {
Ok(None)
} else {
let amt = buf.len();
Ok(Some(buf.split_to(amt)))
}
}
}
impl Encoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn encode(&mut self, item: BytesMut, into: &mut BytesMut) -> io::Result<()> {
into.put(&item[..]);
Ok(())
}
}
#[test]
fn echo() {
drop(env_logger::init());
let pool = CpuPool::new(1);
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
let addr = listener.local_addr().unwrap();
let pool_inner = pool.clone();
let srv = listener.incoming().for_each(move |socket| {
let (sink, stream) = socket.framed(LineCodec).split();
pool_inner.execute(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
Ok(())
});
pool.execute(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
let client = TcpStream::connect(&addr);
let client = client.wait().unwrap();
let (client, _) = write_all(client, b"a\n").wait().unwrap();
let (client, buf, amt) = read(client, vec![0; 1024]).wait().unwrap();
assert_eq!(amt, 2);
assert_eq!(&buf[..2], b"a\n");
let (client, _) = write_all(client, b"\n").wait().unwrap();
let (client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"\n");
let (client, _) = write_all(client, b"b").wait().unwrap();
client.shutdown(Shutdown::Write).unwrap();
let (_client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"b");
}
-88
View File
@@ -1,88 +0,0 @@
#![cfg(unix)]
extern crate env_logger;
extern crate futures;
extern crate libc;
extern crate mio;
extern crate tokio;
extern crate tokio_io;
use std::fs::File;
use std::io::{self, Write};
use std::os::unix::io::{AsRawFd, FromRawFd};
use std::thread;
use std::time::Duration;
use mio::event::Evented;
use mio::unix::{UnixReady, EventedFd};
use mio::{PollOpt, Ready, Token};
use tokio::reactor::{Handle, PollEvented2};
use tokio_io::io::read_to_end;
use futures::Future;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
struct MyFile(File);
impl MyFile {
fn new(file: File) -> MyFile {
unsafe {
let r = libc::fcntl(file.as_raw_fd(), libc::F_SETFL, libc::O_NONBLOCK);
assert!(r != -1, "fcntl error: {}", io::Error::last_os_error());
}
MyFile(file)
}
}
impl io::Read for MyFile {
fn read(&mut self, bytes: &mut [u8]) -> io::Result<usize> {
self.0.read(bytes)
}
}
impl Evented for MyFile {
fn register(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).register(poll, token, interest | hup, opts)
}
fn reregister(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).reregister(poll, token, interest | hup, opts)
}
fn deregister(&self, poll: &mio::Poll) -> io::Result<()> {
EventedFd(&self.0.as_raw_fd()).deregister(poll)
}
}
#[test]
fn hup() {
drop(env_logger::init());
let handle = Handle::default();
unsafe {
let mut pipes = [0; 2];
assert!(libc::pipe(pipes.as_mut_ptr()) != -1,
"pipe error: {}", io::Error::last_os_error());
let read = File::from_raw_fd(pipes[0]);
let mut write = File::from_raw_fd(pipes[1]);
let t = thread::spawn(move || {
write.write_all(b"Hello!\n").unwrap();
write.write_all(b"Good bye!\n").unwrap();
thread::sleep(Duration::from_millis(100));
});
let source = PollEvented2::new_with_handle(MyFile::new(read), &handle).unwrap();
let reader = read_to_end(source, Vec::new());
let (_, content) = t!(reader.wait());
assert_eq!(&b"Hello!\nGood bye!\n"[..], &content[..]);
t.join().unwrap();
}
}
-52
View File
@@ -1,52 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio_io::io;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn basic_runtime_usage() {
let _ = env_logger::init();
// TODO: Don't require the lazy wrapper
tokio::run(::futures::future::lazy(|| {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| println!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| println!("write done"))
.map_err(|e| println!("write err = {:?}", e))
})
})
.map(|_| println!("accept done"));
let client = client
.map_err(|e| println!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| println!("read done"))
.map_err(|e| println!("read err = {:?}", e))
});
tokio::spawn({
server.join(client)
.map(|_| println!("done"))
})
}));
}
-54
View File
@@ -1,54 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::thread;
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::copy;
use tokio_io::AsyncRead;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
let mut s1 = t!(TcpStream::connect(&addr));
let mut s2 = t!(TcpStream::connect(&addr));
let msg = b"foo";
assert_eq!(t!(s1.write(msg)), msg.len());
assert_eq!(t!(s2.write(msg)), msg.len());
let mut buf = [0; 1024];
assert_eq!(t!(s1.read(&mut buf)), msg.len());
assert_eq!(&buf[..msg.len()], msg);
assert_eq!(t!(s2.read(&mut buf)), msg.len());
assert_eq!(&buf[..msg.len()], msg);
});
let future = srv.incoming()
.map(|s| s.split())
.map(|(a, b)| copy(a, b).map(|_| ()))
.buffered(10)
.take(2)
.collect();
t!(future.wait());
t.join().unwrap();
}
-130
View File
@@ -1,130 +0,0 @@
extern crate env_logger;
extern crate tokio;
extern crate mio;
extern crate futures;
use std::{net, thread};
use std::sync::mpsc::channel;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn connect() {
drop(env_logger::init());
let srv = t!(net::TcpListener::bind("127.0.0.1:0"));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
t!(srv.accept()).0
});
let stream = TcpStream::connect(&addr);
let mine = t!(stream.wait());
let theirs = t.join().unwrap();
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
}
#[test]
fn accept() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let (tx, rx) = channel();
let client = srv.incoming().map(move |t| {
tx.send(()).unwrap();
t
}).into_future().map_err(|e| e.0);
assert!(rx.try_recv().is_err());
let t = thread::spawn(move || {
net::TcpStream::connect(&addr).unwrap()
});
let (mine, _remaining) = t!(client.wait());
let mine = mine.unwrap();
let theirs = t.join().unwrap();
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
}
#[test]
fn accept2() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
net::TcpStream::connect(&addr).unwrap()
});
let (tx, rx) = channel();
let client = srv.incoming().map(move |t| {
tx.send(()).unwrap();
t
}).into_future().map_err(|e| e.0);
assert!(rx.try_recv().is_err());
let (mine, _remaining) = t!(client.wait());
mine.unwrap();
t.join().unwrap();
}
#[cfg(unix)]
mod unix {
use tokio::net::TcpStream;
use tokio::prelude::*;
use env_logger;
use futures::future;
use mio::unix::UnixReady;
use std::{net, thread};
use std::time::Duration;
#[test]
fn poll_hup() {
drop(env_logger::init());
let srv = t!(net::TcpListener::bind("127.0.0.1:0"));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
let mut client = t!(srv.accept()).0;
client.write(b"hello world").unwrap();
thread::sleep(Duration::from_millis(200));
});
let mut stream = t!(TcpStream::connect(&addr).wait());
// Poll for HUP before reading.
future::poll_fn(|| {
stream.poll_read_ready(UnixReady::hup().into())
}).wait().unwrap();
// Same for write half
future::poll_fn(|| {
stream.poll_write_ready()
}).wait().unwrap();
let mut buf = vec![0; 11];
// Read the data
future::poll_fn(|| {
stream.poll_read(&mut buf)
}).wait().unwrap();
assert_eq!(b"hello world", &buf[..]);
t.join().unwrap();
}
}
-261
View File
@@ -1,261 +0,0 @@
#![allow(deprecated)]
extern crate futures;
extern crate tokio;
#[macro_use]
extern crate tokio_io;
extern crate bytes;
extern crate env_logger;
use std::io;
use std::net::SocketAddr;
use futures::{Future, Poll, Stream, Sink};
use tokio::net::{UdpSocket, UdpFramed};
use tokio_io::codec::{Encoder, Decoder};
use bytes::{BytesMut, BufMut};
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
fn send_messages<S: SendFn + Clone, R: RecvFn + Clone>(send: S, recv: R) {
let mut a = t!(UdpSocket::bind(&([127, 0, 0, 1], 0).into()));
let mut b = t!(UdpSocket::bind(&([127, 0, 0, 1], 0).into()));
let a_addr = t!(a.local_addr());
let b_addr = t!(b.local_addr());
{
let send = SendMessage::new(a, send.clone(), b_addr, b"1234");
let recv = RecvMessage::new(b, recv.clone(), a_addr, b"1234");
let (sendt, received) = t!(send.join(recv).wait());
a = sendt;
b = received;
}
{
let send = SendMessage::new(a, send, b_addr, b"");
let recv = RecvMessage::new(b, recv, a_addr, b"");
t!(send.join(recv).wait());
}
}
#[test]
fn send_to_and_recv_from() {
send_messages(SendTo {}, RecvFrom {});
}
#[test]
fn send_and_recv() {
send_messages(Send {}, Recv {});
}
trait SendFn {
fn send(&self, &mut UdpSocket, &[u8], &SocketAddr) -> Result<usize, io::Error>;
}
#[derive(Debug, Clone)]
struct SendTo {}
impl SendFn for SendTo {
fn send(&self, socket: &mut UdpSocket, buf: &[u8], addr: &SocketAddr) -> Result<usize, io::Error> {
socket.send_to(buf, addr)
}
}
#[derive(Debug, Clone)]
struct Send {}
impl SendFn for Send {
fn send(&self, socket: &mut UdpSocket, buf: &[u8], addr: &SocketAddr) -> Result<usize, io::Error> {
socket.connect(addr).expect("could not connect");
socket.send(buf)
}
}
struct SendMessage<S> {
socket: Option<UdpSocket>,
send: S,
addr: SocketAddr,
data: &'static [u8],
}
impl<S: SendFn> SendMessage<S> {
fn new(socket: UdpSocket, send: S, addr: SocketAddr, data: &'static [u8]) -> SendMessage<S> {
SendMessage {
socket: Some(socket),
send: send,
addr: addr,
data: data,
}
}
}
impl<S: SendFn> Future for SendMessage<S> {
type Item = UdpSocket;
type Error = io::Error;
fn poll(&mut self) -> Poll<UdpSocket, io::Error> {
let n = try_nb!(self.send.send(self.socket.as_mut().unwrap(), &self.data[..], &self.addr));
assert_eq!(n, self.data.len());
Ok(self.socket.take().unwrap().into())
}
}
trait RecvFn {
fn recv(&self, &mut UdpSocket, &mut [u8], &SocketAddr) -> Result<usize, io::Error>;
}
#[derive(Debug, Clone)]
struct RecvFrom {}
impl RecvFn for RecvFrom {
fn recv(&self, socket: &mut UdpSocket, buf: &mut [u8],
expected_addr: &SocketAddr) -> Result<usize, io::Error> {
socket.recv_from(buf).map(|(s, addr)| {
assert_eq!(addr, *expected_addr);
s
})
}
}
#[derive(Debug, Clone)]
struct Recv {}
impl RecvFn for Recv {
fn recv(&self, socket: &mut UdpSocket, buf: &mut [u8], _: &SocketAddr) -> Result<usize, io::Error> {
socket.recv(buf)
}
}
struct RecvMessage<R> {
socket: Option<UdpSocket>,
recv: R,
expected_addr: SocketAddr,
expected_data: &'static [u8],
}
impl<R: RecvFn> RecvMessage<R> {
fn new(socket: UdpSocket, recv: R, expected_addr: SocketAddr,
expected_data: &'static [u8]) -> RecvMessage<R> {
RecvMessage {
socket: Some(socket),
recv: recv,
expected_addr: expected_addr,
expected_data: expected_data,
}
}
}
impl<R: RecvFn> Future for RecvMessage<R> {
type Item = UdpSocket;
type Error = io::Error;
fn poll(&mut self) -> Poll<UdpSocket, io::Error> {
let mut buf = vec![0u8; 10 + self.expected_data.len() * 10];
let n = try_nb!(self.recv.recv(&mut self.socket.as_mut().unwrap(), &mut buf[..],
&self.expected_addr));
assert_eq!(n, self.expected_data.len());
assert_eq!(&buf[..self.expected_data.len()], &self.expected_data[..]);
Ok(self.socket.take().unwrap().into())
}
}
#[test]
fn send_dgrams() {
let mut a = t!(UdpSocket::bind(&t!("127.0.0.1:0".parse())));
let mut b = t!(UdpSocket::bind(&t!("127.0.0.1:0".parse())));
let mut buf = [0u8; 50];
let b_addr = t!(b.local_addr());
{
let send = a.send_dgram(&b"4321"[..], &b_addr);
let recv = b.recv_dgram(&mut buf[..]);
let (sendt, received) = t!(send.join(recv).wait());
assert_eq!(received.2, 4);
assert_eq!(&received.1[..4], b"4321");
a = sendt.0;
b = received.0;
}
{
let send = a.send_dgram(&b""[..], &b_addr);
let recv = b.recv_dgram(&mut buf[..]);
let received = t!(send.join(recv).wait()).1;
assert_eq!(received.2, 0);
}
}
pub struct ByteCodec;
impl Decoder for ByteCodec {
type Item = Vec<u8>;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<Vec<u8>>, io::Error> {
let len = buf.len();
Ok(Some(buf.split_to(len).to_vec()))
}
}
impl Encoder for ByteCodec {
type Item = Vec<u8>;
type Error = io::Error;
fn encode(&mut self, data: Vec<u8>, buf: &mut BytesMut) -> Result<(), io::Error> {
buf.reserve(data.len());
buf.put(data);
Ok(())
}
}
#[test]
fn send_framed() {
drop(env_logger::init());
let mut a_soc = t!(UdpSocket::bind(&t!("127.0.0.1:0".parse())));
let mut b_soc = t!(UdpSocket::bind(&t!("127.0.0.1:0".parse())));
let a_addr = t!(a_soc.local_addr());
let b_addr = t!(b_soc.local_addr());
{
let a = UdpFramed::new(a_soc, ByteCodec);
let b = UdpFramed::new(b_soc, ByteCodec);
let msg = b"4567".to_vec();
let send = a.send((msg.clone(), b_addr));
let recv = b.into_future().map_err(|e| e.0);
let (sendt, received) = t!(send.join(recv).wait());
let (data, addr) = received.0.unwrap();
assert_eq!(msg, data);
assert_eq!(a_addr, addr);
a_soc = sendt.into_inner();
b_soc = received.1.into_inner();
}
{
let a = UdpFramed::new(a_soc, ByteCodec);
let b = UdpFramed::new(b_soc, ByteCodec);
let msg = b"".to_vec();
let send = a.send((msg.clone(), b_addr));
let recv = b.into_future().map_err(|e| e.0);
let received = t!(send.join(recv).wait()).1;
let (data, addr) = received.0.unwrap();
assert_eq!(msg, data);
assert_eq!(a_addr, addr);
}
}
+19
View File
@@ -0,0 +1,19 @@
# 0.2.0-alpha.1 (August 8, 2019)
### Changed
- Switch to `async`, `await`, and `std::future`.
# 0.1.1 (April 22, 2019)
### Added
- Utilities for creating a `BufStream` from iterators and streams (#1011).
- Add `BufStream::into_stream` (#1048).
- Implement `FromBufStream` for `Bytes` (#1009).
- Implement `Error` for `CollectVecError` (#1010).
### Fixed
- Implement `size_hint` for string types (#1012).
# 0.1.0 (February 23, 2019)
* Initial release
+34
View File
@@ -0,0 +1,34 @@
[package]
name = "tokio-buf"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - Update CHANGELOG.md.
# - Create "v0.2.x" git tag.
version = "0.2.0-alpha.1"
edition = "2018"
authors = ["Tokio Contributors <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-buf/0.2.0-alpha.1/tokio_buf"
description = """
Asynchronous stream of byte buffers
"""
categories = ["asynchronous"]
[dependencies]
bytes = "0.4.10"
either = { version = "1.5", optional = true}
[features]
# default = ["util"]
# util = ["bytes/either", "either"]
[dev-dependencies]
tokio-mock-task = "0.1.1"
[package.metadata.docs.rs]
all-features = true
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+13
View File
@@ -0,0 +1,13 @@
# tokio-buf
Asynchronous stream of byte buffers
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+98
View File
@@ -0,0 +1,98 @@
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.2.0-alpha.1")]
#![warn(
missing_debug_implementations,
missing_docs,
rust_2018_idioms,
unreachable_pub
)]
#![doc(test(no_crate_inject, attr(deny(rust_2018_idioms))))]
//! Asynchronous stream of bytes.
//!
//! This crate contains the `BufStream` trait and a number of combinators for
//! this trait. The trait is similar to `Stream` in the `futures` library, but
//! instead of yielding arbitrary values, it only yields types that implement
//! `Buf` (i.e, byte collections).
// mod never;
mod size_hint;
// mod str;
// mod u8;
// #[cfg(feature = "util")]
// pub mod util;
pub use self::size_hint::SizeHint;
// #[doc(inline)]
// #[cfg(feature = "util")]
// pub use crate::util::BufStreamExt;
use bytes::Buf;
use std::task::{Context, Poll};
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Poll::Pending` means that this stream's next value is not ready yet.
/// Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Poll::Ready(Some(Ok(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Poll::Ready(None)` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Poll::Ready(None)` has been returned,
/// further calls to `poll_buf` may result in a panic or other "bad
/// behavior".
fn poll_buf(&mut self, cx: &mut Context<'_>) -> Poll<Option<Result<Self::Item, Self::Error>>>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
}
+18
View File
@@ -0,0 +1,18 @@
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter<'_>) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter<'_>) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {}
+56
View File
@@ -0,0 +1,56 @@
use std::u64;
/// A `BufStream` size hint
///
/// The default implementation returns:
///
/// * 0 for `available`
/// * 0 for `lower`
/// * `None` for `upper`.
#[derive(Debug, Default, Clone)]
pub struct SizeHint {
lower: u64,
upper: Option<u64>,
}
impl SizeHint {
/// Returns a new `SizeHint` with default values
pub fn new() -> SizeHint {
SizeHint::default()
}
/// Returns the lower bound of data that the `BufStream` will yield before
/// completing.
pub fn lower(&self) -> u64 {
self.lower
}
/// Set the value of the `lower` hint.
///
/// # Panics
///
/// The function panics if `value` is greater than `upper`.
pub fn set_lower(&mut self, value: u64) {
assert!(value <= self.upper.unwrap_or(u64::MAX));
self.lower = value;
}
/// Returns the upper bound of data the `BufStream` will yield before
/// completing, or `None` if the value is unknown.
pub fn upper(&self) -> Option<u64> {
self.upper
}
/// Set the value of the `upper` hint value.
///
/// # Panics
///
/// This function panics if `value` is less than `lower`.
pub fn set_upper(&mut self, value: u64) {
// There is no need to check `available` as that is guaranteed to be
// less than or equal to `lower`.
assert!(value >= self.lower, "`value` is less than than `lower`");
self.upper = Some(value);
}
}
+55
View File
@@ -0,0 +1,55 @@
use crate::never::Never;
use crate::BufStream;
use crate::SizeHint;
use futures::Poll;
use std::io;
use std::mem;
impl BufStream for String {
type Item = io::Cursor<Vec<u8>>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
let bytes = mem::replace(self, Default::default()).into_bytes();
let buf = io::Cursor::new(bytes);
Ok(Some(buf).into())
}
fn size_hint(&self) -> SizeHint {
size_hint(&self[..])
}
}
impl BufStream for &'static str {
type Item = io::Cursor<&'static [u8]>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
let bytes = mem::replace(self, Default::default()).as_bytes();
let buf = io::Cursor::new(bytes);
Ok(Some(buf).into())
}
fn size_hint(&self) -> SizeHint {
size_hint(&self[..])
}
}
fn size_hint(s: &str) -> SizeHint {
let mut hint = SizeHint::new();
hint.set_lower(s.len() as u64);
hint.set_upper(s.len() as u64);
hint
}
+66
View File
@@ -0,0 +1,66 @@
use crate::never::Never;
use crate::BufStream;
use bytes::{Bytes, BytesMut};
use futures::Poll;
use std::io;
impl BufStream for Vec<u8> {
type Item = io::Cursor<Vec<u8>>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
impl BufStream for &'static [u8] {
type Item = io::Cursor<&'static [u8]>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
impl BufStream for Bytes {
type Item = io::Cursor<Bytes>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
impl BufStream for BytesMut {
type Item = io::Cursor<BytesMut>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
fn poll_bytes<T: Default>(buf: &mut T) -> Poll<Option<io::Cursor<T>>, Never> {
use std::mem;
let bytes = mem::replace(buf, Default::default());
let buf = io::Cursor::new(bytes);
Ok(Some(buf).into())
}
+46
View File
@@ -0,0 +1,46 @@
use crate::BufStream;
use either::Either;
use futures::{try_ready, Poll};
/// A buf stream that sequences two buf streams together.
///
/// `Chain` values are produced by the `chain` function on `BufStream`.
#[derive(Debug)]
pub struct Chain<T, U> {
left: Option<T>,
right: U,
}
impl<T, U> Chain<T, U> {
pub(crate) fn new(left: T, right: U) -> Chain<T, U> {
Chain {
left: Some(left),
right,
}
}
}
impl<T, U> BufStream for Chain<T, U>
where
T: BufStream,
U: BufStream<Error = T::Error>,
{
type Item = Either<T::Item, U::Item>;
type Error = T::Error;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if let Some(ref mut stream) = self.left {
let res = try_ready!(stream.poll_buf());
if res.is_some() {
return Ok(res.map(Either::Left).into());
}
}
self.left = None;
let res = try_ready!(self.right.poll_buf());
Ok(res.map(Either::Right).into())
}
}
+101
View File
@@ -0,0 +1,101 @@
use super::FromBufStream;
use crate::BufStream;
use futures::{try_ready, Future, Poll};
/// Consumes a buf stream, collecting the data into a single byte container.
///
/// `Collect` values are produced by `BufStream::collect`.
#[derive(Debug)]
pub struct Collect<T, U>
where
T: BufStream,
U: FromBufStream<T::Item>,
{
stream: T,
builder: Option<U::Builder>,
}
/// Errors returned from `Collect` future.
#[derive(Debug)]
pub struct CollectError<T, U> {
inner: Error<T, U>,
}
#[derive(Debug)]
enum Error<T, U> {
Stream(T),
Collect(U),
}
impl<T, U> Collect<T, U>
where
T: BufStream,
U: FromBufStream<T::Item>,
{
pub(crate) fn new(stream: T) -> Collect<T, U> {
let builder = U::builder(&stream.size_hint());
Collect {
stream,
builder: Some(builder),
}
}
}
impl<T, U> Future for Collect<T, U>
where
T: BufStream,
U: FromBufStream<T::Item>,
{
type Item = U;
type Error = CollectError<T::Error, U::Error>;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
loop {
let res = self.stream.poll_buf().map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
match try_ready!(res) {
Some(mut buf) => {
let builder = self.builder.as_mut().expect("cannot poll after done");
U::extend(builder, &mut buf, &self.stream.size_hint()).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
}
None => {
let builder = self.builder.take().expect("cannot poll after done");
let value = U::build(builder).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
return Ok(value.into());
}
}
}
}
}
// ===== impl CollectError =====
impl<T, U> CollectError<T, U> {
/// Returns `true` if the error was caused by polling the stream.
pub fn is_stream_err(&self) -> bool {
match self.inner {
Error::Stream(_) => true,
_ => false,
}
}
/// Returns `true` if the error happened while collecting the data.
pub fn is_collect_err(&self) -> bool {
match self.inner {
Error::Collect(_) => true,
_ => false,
}
}
}
+154
View File
@@ -0,0 +1,154 @@
use crate::SizeHint;
use bytes::{Buf, BufMut, Bytes};
use std::error::Error;
use std::fmt;
use std::usize;
/// Conversion from a `BufStream`.
///
/// By implementing `FromBufStream` for a type, you define how it will be
/// created from a buf stream. This is common for types which describe byte
/// storage of some kind.
///
/// `FromBufStream` is rarely called explicitly, and it is instead used through
/// `BufStream`'s `collect` method.
pub trait FromBufStream<T: Buf>: Sized {
/// Type that is used to build `Self` while the `BufStream` is being
/// consumed.
type Builder;
/// Error that might happen on conversion.
type Error;
/// Create a new, empty, builder. The provided `hint` can be used to inform
/// reserving capacity.
fn builder(hint: &SizeHint) -> Self::Builder;
/// Extend the builder with the `Buf`.
///
/// This method is called whenever a new `Buf` value is obtained from the
/// buf stream.
///
/// The provided size hint represents the state of the stream **after**
/// `buf` has been yielded. The lower bound represents the minimum amount of
/// data that will be provided after this call to `extend` returns.
fn extend(builder: &mut Self::Builder, buf: &mut T, hint: &SizeHint)
-> Result<(), Self::Error>;
/// Finalize the building of `Self`.
///
/// Called once the buf stream is fully consumed.
fn build(builder: Self::Builder) -> Result<Self, Self::Error>;
}
/// Error returned from collecting into a `Vec<u8>`
#[derive(Debug)]
pub struct CollectVecError {
_p: (),
}
/// Error returned from collecting into a `Bytes`
#[derive(Debug)]
pub struct CollectBytesError {
_p: (),
}
impl<T: Buf> FromBufStream<T> for Vec<u8> {
type Builder = Vec<u8>;
type Error = CollectVecError;
fn builder(hint: &SizeHint) -> Vec<u8> {
Vec::with_capacity(hint.lower() as usize)
}
fn extend(builder: &mut Self, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
let lower = hint.lower();
// If the lower bound is greater than `usize::MAX` then we have a
// problem
if lower > usize::MAX as u64 {
return Err(CollectVecError { _p: () });
}
let mut reserve = lower as usize;
// If `upper` is set, use this value if it is less than or equal to 64.
// This only really impacts the first iteration.
match hint.upper() {
Some(upper) if upper <= 64 => {
reserve = upper as usize;
}
_ => {}
}
// hint.lower() represents the minimum amount of data that will be
// received *after* this function call. We reserve this amount on top of
// the amount of data in `buf`.
reserve = match reserve.checked_add(buf.remaining()) {
Some(n) => n,
None => return Err(CollectVecError { _p: () }),
};
// Always reserve 64 bytes the first time, unless `upper` is set and is
// less than 64.
if builder.is_empty() {
reserve = reserve.max(match hint.upper() {
Some(upper) if upper < 64 => upper as usize,
_ => 64,
});
}
// Make sure overflow won't happen when reserving
if reserve.checked_add(builder.len()).is_none() {
return Err(CollectVecError { _p: () });
}
// Reserve space
builder.reserve(reserve);
// Copy the data
builder.put(buf);
Ok(())
}
fn build(builder: Self) -> Result<Self, Self::Error> {
Ok(builder)
}
}
impl<T: Buf> FromBufStream<T> for Bytes {
type Builder = Vec<u8>;
type Error = CollectBytesError;
fn builder(hint: &SizeHint) -> Vec<u8> {
<Vec<u8> as FromBufStream<T>>::builder(hint)
}
fn extend(builder: &mut Vec<u8>, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
<Vec<u8> as FromBufStream<T>>::extend(builder, buf, hint)
.map_err(|_| CollectBytesError { _p: () })
}
fn build(builder: Vec<u8>) -> Result<Self, Self::Error> {
Ok(builder.into())
}
}
impl fmt::Display for CollectVecError {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(fmt, "BufStream is too big")
}
}
impl Error for CollectVecError {}
impl fmt::Display for CollectBytesError {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(fmt, "BufStream too big")
}
}
impl Error for CollectBytesError {}
+50
View File
@@ -0,0 +1,50 @@
use crate::BufStream;
use bytes::Buf;
use futures::Poll;
use std::error::Error;
use std::fmt;
/// Converts an `Iterator` into a `BufStream` which is always ready to yield the
/// next value.
///
/// Iterators in Rust don't express the ability to block, so this adapter
/// simply always calls `iter.next()` and returns that.
pub fn iter<I>(i: I) -> Iter<I::IntoIter>
where
I: IntoIterator,
I::Item: Buf,
{
Iter {
iter: i.into_iter(),
}
}
/// `BufStream` returned by the [`iter`] function.
#[derive(Debug)]
pub struct Iter<I> {
iter: I,
}
#[derive(Debug)]
pub enum Never {}
impl<I> BufStream for Iter<I>
where
I: Iterator,
I::Item: Buf,
{
type Item = I::Item;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
Ok(self.iter.next().into())
}
}
impl fmt::Display for Never {
fn fmt(&self, _: &mut fmt::Formatter<'_>) -> fmt::Result {
unreachable!();
}
}
impl Error for Never {}
+76
View File
@@ -0,0 +1,76 @@
use crate::BufStream;
use bytes::Buf;
use futures::Poll;
/// Limits the stream to a maximum amount of data.
#[derive(Debug)]
pub struct Limit<T> {
stream: T,
remaining: u64,
}
/// Errors returned from `Limit`.
#[derive(Debug)]
pub struct LimitError<T> {
/// When `None`, limit was reached
inner: Option<T>,
}
impl<T> Limit<T> {
pub(crate) fn new(stream: T, amount: u64) -> Limit<T> {
Limit {
stream,
remaining: amount,
}
}
}
impl<T> BufStream for Limit<T>
where
T: BufStream,
{
type Item = T::Item;
type Error = LimitError<T::Error>;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
use futures::Async::Ready;
if self.stream.size_hint().lower() > self.remaining {
return Err(LimitError { inner: None });
}
let res = self
.stream
.poll_buf()
.map_err(|err| LimitError { inner: Some(err) });
match res {
Ok(Ready(Some(ref buf))) => {
if buf.remaining() as u64 > self.remaining {
self.remaining = 0;
return Err(LimitError { inner: None });
}
self.remaining -= buf.remaining() as u64;
}
_ => {}
}
res
}
}
// ===== impl LimitError =====
impl<T> LimitError<T> {
/// Returns `true` if the error was caused by polling the stream.
pub fn is_stream_err(&self) -> bool {
self.inner.is_some()
}
/// Returns `true` if the stream reached its limit.
pub fn is_limit_err(&self) -> bool {
self.inner.is_none()
}
}
+87
View File
@@ -0,0 +1,87 @@
//! Types and utilities for working with `BufStream`.
mod chain;
mod collect;
mod from;
mod iter;
mod limit;
mod stream;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::iter::iter;
pub use self::limit::Limit;
pub use self::stream::{stream, IntoStream};
pub mod error {
//! Error types
pub use super::collect::CollectError;
pub use super::from::{CollectBytesError, CollectVecError};
pub use super::limit::LimitError;
}
use crate::BufStream;
impl<T> BufStreamExt for T where T: BufStream {}
/// An extension trait for `BufStream`'s that provides a variety of convenient
/// adapters.
pub trait BufStreamExt: BufStream {
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
/// Creates a `Stream` from a `BufStream`.
///
/// This produces a `Stream` of `BufStream::Items`.
fn into_stream(self) -> IntoStream<Self>
where
Self: Sized,
{
IntoStream::new(self)
}
}
+76
View File
@@ -0,0 +1,76 @@
use crate::BufStream;
use bytes::Buf;
use futures::{Async, Poll, Stream};
/// Converts a `Stream` of `Buf` types into a `BufStream`.
///
/// While `Stream` and `BufStream` are very similar, they are not identical. The
/// `stream` function returns a `BufStream` that is backed by the provided
/// `Stream` type.
pub fn stream<T>(stream: T) -> FromStream<T>
where
T: Stream,
T::Item: Buf,
{
FromStream { stream }
}
/// `BufStream` returned by the [`stream`] function.
#[derive(Debug)]
pub struct FromStream<T> {
stream: T,
}
impl<T> BufStream for FromStream<T>
where
T: Stream,
T::Item: Buf,
{
type Item = T::Item;
type Error = T::Error;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
self.stream.poll()
}
}
/// Converts a `BufStream` into a `Stream`.
#[derive(Debug)]
pub struct IntoStream<T> {
buf: T,
}
impl<T> IntoStream<T> {
/// Create a new `Stream` from the provided `BufStream`.
pub fn new(buf: T) -> Self {
IntoStream { buf }
}
/// Get a reference to the inner `BufStream`.
pub fn get_ref(&self) -> &T {
&self.buf
}
/// Get a mutable reference to the inner `BufStream`
pub fn get_mut(&mut self) -> &mut T {
&mut self.buf
}
/// Get the inner `BufStream`.
pub fn into_inner(self) -> T {
self.buf
}
}
impl<T: BufStream> Stream for IntoStream<T> {
type Item = T::Item;
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.buf.poll_buf()? {
Async::Ready(Some(buf)) => Ok(Async::Ready(Some(buf))),
Async::Ready(None) => Ok(Async::Ready(None)),
Async::NotReady => Ok(Async::NotReady),
}
}
}
+8
View File
@@ -0,0 +1,8 @@
use tokio_buf::BufStream;
#[test]
fn object_safe() {
// Ensures that `BufStream` can be a trait object
#[allow(dead_code)]
fn obj(_: &mut dyn BufStream<Item = u32, Error = ()>) {}
}
+38
View File
@@ -0,0 +1,38 @@
#![cfg(feature = "broken")]
#![cfg(feature = "util")]
use futures::Async::*;
use tokio_buf::{BufStream, BufStreamExt};
mod support;
use support::*;
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"]).chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[Ok(Ready("foo")), Ok(NotReady), Ok(Ready("bar"))]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
+63
View File
@@ -0,0 +1,63 @@
#![cfg(feature = "broken")]
#![cfg(feature = "util")]
use bytes::Bytes;
use futures::Future;
use tokio_buf::BufStreamExt;
mod support;
use support::*;
macro_rules! test_collect_impl {
($t:ty $(, $capacity:ident)*) => {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world, one two three"[..]);
}
}
#[test]
fn collect_vec() {
test_collect_impl!(Vec<u8>, capacity);
}
#[test]
fn collect_bytes() {
test_collect_impl!(Bytes);
}
+29
View File
@@ -0,0 +1,29 @@
#![cfg(feature = "broken")]
use futures::Async::*;
use std::io::Cursor;
use tokio_buf::{util, BufStream};
mod support;
type Buf = Cursor<&'static [u8]>;
#[test]
fn empty_iter() {
let mut bs = util::iter(Vec::<Buf>::new());
assert_none!(bs.poll_buf());
}
#[test]
fn full_iter() {
let bufs = vec![buf(b"one"), buf(b"two"), buf(b"three")];
let mut bs = util::iter(bufs);
assert_buf_eq!(bs.poll_buf(), "one");
assert_buf_eq!(bs.poll_buf(), "two");
assert_buf_eq!(bs.poll_buf(), "three");
assert_none!(bs.poll_buf());
}
fn buf(data: &'static [u8]) -> Buf {
Cursor::new(data)
}
+60
View File
@@ -0,0 +1,60 @@
#![cfg(feature = "broken")]
#![cfg(feature = "util")]
use futures::Async::*;
use futures::Future;
use tokio_buf::{BufStream, BufStreamExt};
mod support;
use support::*;
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world").limit(5).collect::<Vec<_>>().wait();
assert!(res.is_err());
let res = one("hello world").limit(10).collect::<Vec<_>>().wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"]).limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
+40
View File
@@ -0,0 +1,40 @@
use tokio_buf::SizeHint;
#[test]
fn size_hint() {
let hint = SizeHint::new();
assert_eq!(hint.lower(), 0);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_lower(100);
assert_eq!(hint.lower(), 100);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_upper(200);
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(200));
let mut hint = SizeHint::new();
hint.set_lower(100);
hint.set_upper(100);
assert_eq!(hint.lower(), 100);
assert_eq!(hint.upper(), Some(100));
}
#[test]
#[should_panic]
fn size_hint_lower_bigger_than_upper() {
let mut hint = SizeHint::new();
hint.set_upper(100);
hint.set_lower(200);
}
#[test]
#[should_panic]
fn size_hint_upper_less_than_lower() {
let mut hint = SizeHint::new();
hint.set_lower(200);
hint.set_upper(100);
}
+44
View File
@@ -0,0 +1,44 @@
#![cfg(feature = "broken")]
use futures::sync::mpsc;
use futures::Async::*;
use std::io::Cursor;
use tokio_buf::{util, BufStream};
use tokio_mock_task::MockTask;
mod support;
type Buf = Cursor<&'static [u8]>;
#[test]
fn empty_stream() {
let (_, rx) = mpsc::unbounded::<Buf>();
let mut bs = util::stream(rx);
assert_none!(bs.poll_buf());
}
#[test]
fn full_stream() {
let (tx, rx) = mpsc::unbounded();
let mut bs = util::stream(rx);
let mut task = MockTask::new();
tx.unbounded_send(buf(b"one")).unwrap();
assert_buf_eq!(bs.poll_buf(), "one");
task.enter(|| assert_not_ready!(bs.poll_buf()));
tx.unbounded_send(buf(b"two")).unwrap();
assert!(task.is_notified());
assert_buf_eq!(bs.poll_buf(), "two");
task.enter(|| assert_not_ready!(bs.poll_buf()));
drop(tx);
assert!(task.is_notified());
assert_none!(bs.poll_buf());
}
fn buf(data: &'static [u8]) -> Buf {
Cursor::new(data)
}

Some files were not shown because too many files have changed in this diff Show More