Compare commits

..
Author SHA1 Message Date
Carl Lerche c25ea78ec9 Bump version of a number of sub crates (#414)
This includes:

* tokio-codec (0.1.0)
* tokio-current-thread (0.1.0)
* tokio-fs (0.1.1)
* tokio-io (0.1.7)
* tokio-reactor (0.1.2)
* tokio-udp (0.1.1)
2018-06-13 10:24:56 -07:00
Carl Lerche 2e0cd292d2 Fix some broken doc links (#413) 2018-06-13 09:02:46 -07:00
Sylwek 4ebaf18c27 Typo (#415) 2018-06-13 09:02:34 -07:00
Carl Lerche ab07733d66 Deprecate executor re-exports (#412) 2018-06-12 14:41:12 -07:00
Mat Sadler d1f825ca13 Add OpenOptions to tokio-fs (#390)
Add an `OpenOptions` struct to `tokio-fs` that mirrors the one found in
`std`. Also provide a conversion from a `std` instance to a Tokio instance.
2018-06-12 10:47:24 -07:00
Laurențiu Nicola 4cf7d73b22 tokio-fs: add into_std (#403) 2018-06-12 10:40:43 -07:00
jpbriquet 2cd854c2c7 tokio-current-thread crate (#370)
Extract `tokio::executor::current_thread` to a tokio-current-thread
crate. Deprecated fns stay in the old location. The new crate only
contains thee most recent API.
2018-06-12 10:26:03 -07:00
Carl Lerche ba05c39d65 Fix a deadlock that can happen when shutting down (#409)
There is a deadlock that can occur when the concurrent runtime shuts
down. This patch adds a test and fix.

Fixes #401.
2018-06-12 09:41:18 -07:00
Alyssa Ross 64b8884911 Fix typo in comment (#402) 2018-06-11 15:26:55 -07:00
pravic d391e63418 Duplicated word in documentation. (#405) 2018-06-11 15:17:09 -07:00
Carl Lerche 8d8c895a1c Remove tokio-codec dependency from tokio (#397)
This will be added again later once types are re-exported.
2018-06-08 09:56:40 -07:00
Carl Lerche dba5c27296 Bump version to v0.1.7 (#396)
This also bumps the versions of:

* tokio-threadpool
* tokio-timer
2018-06-06 20:14:35 -07:00
Carl Lerche db620b42ec Another attempt at abstracting Instant::now (#381)
Currently, the timer uses a `Now` trait to abstract the source of time.
This allows time to be mocked out. However, the current implementation
has a number of limitations as represented by #288 and #296.

The main issues are that `Now` requires `&mut self` which prevents a
value from being easily used in a concurrent environment. Also, when
wanting to write code that is abstract over the source of time, generics
get out of hand.

This patch provides an alternate solution. A new type, `Clock` is
provided which defaults to `Instant::now` as the source of time, but
allows configuring the actual source using a new iteration of the `Now`
trait. This time, `Now` is `Send + Sync + 'static`. Internally, `Clock`
stores the now value in an `Arc<Now>` value, which introduces dynamism
and allows `Clock` values to be cloned and be `Sync`.

Also, the current clock can be set for the current execution context
using the `with_default` pattern.

Because using the `Instant::now` will be the most common case by far, it
is special cased in order to avoid the need to allocate an `Arc` and use
dynamic dispatch.
2018-06-06 16:04:39 -07:00
David Kellum 9013ed9bd4 Fix description of BlockingError as io::Error (#384) 2018-06-06 14:34:55 -07:00
Carl Lerche 06325fa63b Bump tokio-uds to v0.2.0 (#395) 2018-06-06 14:09:07 -07:00
Sebastian Dröge 0d41ba7a08 Implement a Send Handle for the single-threaded Runtime (#340)
Implement a Send'able Handle for the single-threaded `Runtime` and
`CurrentThread` executor to spawn new tasks from other threads.
2018-06-05 16:56:15 -07:00
Carl Lerche c07a7b26d3 Cleanup FramedParts in new tokio-codec (#394) 2018-06-05 15:31:01 -07:00
Bryan Burgers f723d10087 Create tokio-codec (#360)
Create a new tokio-codec crate with many of the contents of
`tokio_io::codec`.
2018-06-04 20:36:06 -07:00
Jon Gjengset 3d7263d3a0 Implement Runtime::block_on using oneshot (#391) 2018-06-04 20:09:17 -07:00
Carl Lerche 9caec1c15d Remove futures2 crate (#380) 2018-05-29 16:28:00 -07:00
Carl Lerche 703f07ca17 Remove threadpool disclaimer (#378) 2018-05-29 15:59:37 -07:00
Michal 'vorner' Vaner db9371126d Include a manually built runtime example (#306) 2018-05-29 14:44:28 -07:00
Carl Lerche eb1cf8fc9b Unpin Rust nightly version (#379) 2018-05-29 14:36:52 -07:00
Carl Lerche 4af6109398 Fix bug related to spawning optimization (#375)
The thread pool optimizes cases where a task currently running on the
pool spawns a new future. However, the optimization did not factor in
cases where two thread pools interacted.

This patch fixes the optimization and includes a test.

Fixes #342
2018-05-24 22:06:32 -07:00
Roman Zeyde 96f3ec903c Fix a small typo in README.md (#373) 2018-05-23 12:07:46 -07:00
Chris Pick 8c791fd0bf Fix Runtime::new's doc link to tokio::run (#371) 2018-05-22 15:29:15 -07:00
Rijenkii c0747a5fc1 tokio-io: Fix the link to the repository (#372) 2018-05-22 15:28:28 -07:00
Carl Lerche c8e710d39e Import tokio-uds (#365)
This imports tokio-uds from the dedicated repo.
2018-05-14 14:48:32 -07:00
Carl Lerche e281e4f4cb Remove fuchsia references as it is not supported. (#355) 2018-05-14 12:00:19 -07:00
Carl Lerche 6598334021 Add Gitter badge to README (#358) 2018-05-14 12:00:10 -07:00
main() 35f3351c97 Document Handle::default() behavior (#359) 2018-05-14 11:11:28 -07:00
Jason Davies 1f5bb121e2 Fix typo in doc comment. (#361) 2018-05-14 11:10:25 -07:00
sbstp 88801bb613 timer: add sleep free function (#347) 2018-05-11 09:16:08 -07:00
Carl Lerche a850063211 Handle::default() should lazily bind to reactor. (#350)
Currently, not specifying a `Handle` is different than using
`Handle::default()`. This is because `Handle::default()` will
immediately bind to the reactor for the current context vs. not
specifying a `Handle`, which binds to a reactor when it is polled.

This patch changes the `Handle::default()` behavior, bringing it inline
with actual defaults.

`Handle::current()` still immediately binds to the current reactor.

Fixes #307
2018-05-11 08:32:03 -07:00
Marek Kotewicz 14ec268b8a Fixed broken link in tokio-fs documentation (#352) 2018-05-11 08:31:06 -07:00
Thijs Vermeir 363b207f2b Fix typo in documentation (#346) 2018-05-08 11:44:50 -07:00
Julian Tescher 06b2c40222 Fix typos (#348) 2018-05-08 11:44:17 -07:00
Thijs Vermeir 68b82f5721 Fix typo in documentation (#341) 2018-05-04 07:06:47 -07:00
Thijs Vermeir 7cca6499a9 Fix typo in documentation (#338) 2018-05-03 10:28:48 -07:00
Carl Lerche 8235eefbf0 Fix some dependency versions (#337) 2018-05-02 13:12:33 -07:00
Carl Lerche 14b31bdba5 Bump version to v0.1.6 (#336) 2018-05-02 12:14:44 -07:00
Carl Lerche f768163982 Filesystem manipulation APIs. (#323)
This patch adds a new crate: tokio-fs. This crate provides a wrapper
around `std` functionality that can only be performed using blocking
operations. This primarily includes filesystem operations, but it also
includes standard input, output, and error access as these streams
cannot be safely switched to non-blocking mode in a portable way.

These wrappers call the `std` functions from within a `blocking`
annotation which allows the runtime to compensate for the fact that the
thread will potentially remain blocked in a system call.
2018-05-02 11:19:58 -07:00
Carl Lerche 7a2b5db15c Remove futures2 feature from Cargo.toml files (#334)
Currently, the state of the futures2 integration is pretty broken. This
patch removes the feature flag, preventing users from trying to use it.
In the future, it can be brought back when the implementation is fixed.
2018-05-02 10:48:58 -07:00
Roman 2465483845 Current thread runtime (#308)
This patch introduces a version of `Runtime` that runs all components on
the current thread. This allows users to spawn futures that do not implement
`Send`.
2018-05-02 09:40:42 -07:00
Stefan Bühler 6a0ecef81a Timer: always park nested Park (#327)
The nested `Park` might need to do some work, even if the duration is 0
seconds (e.g. a `Reactor`).

Similar to what #313 did for CurrentThread.
2018-05-01 16:33:41 -07:00
Stefan Bühler 6defeeb2ba current_thread: make underlying Park instance accessible 2018-05-01 14:33:38 -07:00
Stefan Bühler b36a73059d tokio-io: require bytes-0.4.7 for Buf::get_uint_be 2018-05-01 14:33:38 -07:00
Roman d1d4fe4d07 Stop using deprecated bytes APIs in tests (#324) (#331) 2018-04-30 10:02:48 -07:00
Carl Lerche 9aaa8f06d1 Stop using deprecated bytes APIs (#324)
This also adds a filter for another treiber stack expected data race. The
race is expected as part of the algorithm.
2018-04-28 12:25:22 -07:00
Sebastian Dröge 6ea00162b9 Make CurrentThread::turn() more fair by always parking with 0 timeout… (#313)
This ensures that all fd-based futures are put into the queue for the
current tick, if the CurrentThread is parking via the Reactor.

Otherwise, if there are queued up futures already, only those would be
polled in the turn. These futures could then notify others/themselves to
have the queue still non-empty on the next turn. Which then potentially
allows the reactor to never be polled, and thus fd-based futures are
never queued up and polled.

Also return in the Turn return value whether any futures were polled at
all, which allows the caller to know if any work was done at all in this
turn and based on that adjust behavior.
2018-04-25 10:37:18 -07:00
Carl Lerche 61d635e8ad Threadpool blocking (#317)
This patch adds a `blocking` to `tokio-threadpool`. This function serves
as a way to annotate sections of code that will perform blocking
operations. This informs the thread pool that an additional thread needs
to be spawned to replace the current thread, which will no longer be
able to process the work queue.
2018-04-15 12:29:22 -07:00
Carl Lerche 372400ed34 Add additional timer::Error docs. (#311)
Closes #302
2018-04-10 14:28:37 -07:00
Roman ba9d849ef0 Fix warning: variable does not need to be mutable (#309) 2018-04-10 13:33:05 -07:00
Roman 5b677934fe Add example that prints each packet from tcp client (#301) 2018-04-10 13:08:55 -07:00
Sam Rijs dbcd8353b0 Update futures2 to use the futures 0.2 release (#304) 2018-04-08 20:23:33 -07:00
Carl Lerche 3be6b69e1b Refactor threadpool task types (#300)
Replaces homegrown Arc with std Arc

Is this safer? Unknown. At least we don't have to maintain an arc
implementation anymore. This will also make it easier to filter out tsan
false positives.

Also split task/mod.rs into multiple files.
2018-04-05 10:57:05 -07:00
Carl Lerche 0bcf9b0ae6 ThreadPool refactoring (#299) 2018-04-04 13:30:54 -07:00
Carl Lerche c715739599 Add arc::Weak to tsan filter. (#298) 2018-04-04 12:54:49 -07:00
David 6aea9c43e8 Update Cargo.toml (#293) 2018-04-04 09:18:56 -07:00
Igor Gnatenko 82f6a52d1a threadpool: bump minimal version of executor (#292) 2018-04-04 09:18:40 -07:00
Leandro Pacheco a6b307cfbe re-export io::{ReadHalf/WriteHalf} timer::Error (#290) 2018-04-04 09:18:12 -07:00
Roman dcb20b289c Build 32/64-bit Linux and FreeBSD on Travis CI (#286) 2018-04-04 08:37:28 -07:00
Carl Lerche 79afc7ee68 Threadpool refactor (#294)
* Switch worker lifecycle to an enum
* Move some files around
* Rename State -> PoolState
2018-04-03 22:35:59 -07:00
Kam Y. Tse 3ba5595233 Fix typo (#275) 2018-04-02 13:11:06 -07:00
Carl Lerche 7232ba6d55 Bump tokio-timer to v0.2.1 (#287) 2018-04-02 11:06:22 -07:00
Roman a14de909eb Build both x86 and x64 on Windows (#282) 2018-04-02 09:37:56 -07:00
laizy d8789cd379 fix panic in chat example (#279) 2018-04-02 09:00:40 -07:00
Roman 8d4be0361e Fix unused variable in tokio-threadpool\tests\threadpool.rs:581:9 (#284) 2018-04-02 09:00:14 -07:00
Daniel Griffen 3f2710397d Fix tokio-timer on 32bit systems (#274) 2018-04-02 08:53:43 -07:00
Roman 8895a7d3ab Fix Appveyor badge on crates.io page (#280) 2018-04-01 16:16:22 -07:00
Carl Lerche 10cb9dd468 Actually bump tokio to v0.1.5 (#273) 2018-03-30 15:37:52 -07:00
Carl Lerche 2ca214bd2c Fix tokio dependency versions (#272) 2018-03-30 15:32:25 -07:00
Carl Lerche 2a01c26d58 Bump version to v0.1.5 (#271)
This also bumps:

* tokio-executor to v0.1.2
* tokio-threadpool to v0.1.2
* tokio-timer to v0.2.0
2018-03-30 15:28:44 -07:00
Carl Lerche ea172537aa Rename Sleep to Delay (#270)
This patch renames `Sleep` from tokio-timer and the tokio facade to
`Delay`. Given that the future does not actually put anything to sleep,
the `Delay` name feels more appropriate.

Fixes #263
2018-03-30 14:21:48 -07:00
Carl Lerche baa2502ec6 Integrate timers with runtime. (#266)
This patch integrate the new timer implementation with the runtime by
initializing a timer per worker thread. This allows minimizing the
amount of synchronization needed for using timers.
2018-03-30 11:50:02 -07:00
Carl Lerche d4d17392fe Remove println from tests (#267) 2018-03-29 20:58:02 -07:00
Carl Lerche 6807363efd Fix permissions (#268) 2018-03-29 20:42:59 -07:00
Carl Lerche 1c5d131245 Allow customizing the threadpool's parker (#264)
* Allow customizing the threadpool's parker

This patch allows the user of threadpool to customize how the worker
threads park themselves. This allows custom parking logic to be
injected. For example, this allows embedding a timer on each worker
thread.

* Call `park` instance every so often.

Since the `park` is now customizable, it might have logic that must be
called every so often. For example, a timer might have timeouts that it
must expire.

Currently, if a worker is very busy, it won't call into the `park`
instance. This patch changes this so that after every 32 task
invocations, `park` is called with a duration of zero.
2018-03-29 13:47:08 -07:00
Carl Lerche 19500f7df8 Provide a timer implementation (#249)
This patch adds a new crate: tokio-timer. This crate provides an
efficient timer implemeentation designed for use in Tokio based
applications.

The timer users a hierarchical hashed timer wheel algorithm with six
levels, each having 64 slots. This allows the timer to have a resolution
of 1ms while maintaining O(1) complexity for insert, removal, and firing
of timeouts.

There already exists a tokio-timer crate. This is a complete rewrite
which solves the outstanding problems with the existing tokio-timer
library.

Closes #146.
2018-03-28 22:26:47 -07:00
Roman ad189826f4 Split tokio-threadpool lib.rs into files (#233)
* Builder -> src/builder.rs
* Callback -> src/callback.rs
* Config -> src/config.rs
* Futures2Wake -> src/futures2_wake.rs
* Inner -> src/inner.rs
* Notifier-> src/notifier.rs
* Sender -> src/sender.rs
* Shutdown -> src/shutdown.rs
* ShutdownTask -> src/shutdown_task.rs
* SleepStack -> src/sleep_stack.rs
* State -> src/state.rs
* ThreadPool -> src/thread_pool.rs
* Worker -> src/worker.rs
* WorkerEntry -> src/worker_entry.rs
* WorkerState -> src/worker_state.rs
2018-03-27 15:56:21 -07:00
Klaus Purer a612736f54 fix(cargo): Bump dependencies so that Tokio compiles with minimal versions (#258) 2018-03-27 15:47:11 -07:00
Carl Lerche bda8dd5113 Update futures2 Cargo.toml (#256) 2018-03-27 15:46:19 -07:00
Sam Rijs 415a786049 Fix unstable-futures feature flag propagation (#261) 2018-03-27 15:46:02 -07:00
Carl Lerche 2edc35a45d Disable future 0.2 tests (#259) 2018-03-24 14:01:41 -07:00
Carl Lerche 9cffda59c9 Bump version to v0.1.4 (#252)
This also bumps:

* tokio-executor: v0.1.1
* tokio-reactor: v0.1.1
* tokio-threadpool: v0.1.1
2018-03-23 10:34:42 -07:00
Carl Lerche 23d95b44e7 Change hammer test consts (#255) 2018-03-23 10:27:58 -07:00
Denis 69e45f4be4 Fix a typo in tokio_reactor::PollEvented (#250)
`read_readiness` -> `write_readiness` in *write* methods
2018-03-22 20:15:31 -07:00
Denis 6bdfa159a7 Fix connect example for UDP (#241)
Close #241
2018-03-22 10:02:01 -07:00
Michal 'vorner' Vaner 5d87a9cee1 Docs: warn about errors from TcpListener::incoming (#247) 2018-03-22 10:01:05 -07:00
Denis 16d3540ce9 Add UDP client example (send/recv_dgram) (#239) 2018-03-22 10:00:27 -07:00
Sean McArthur e5ebd02885 implement poll_vectored_* and initializer method for futures2 (#242) 2018-03-22 09:59:13 -07:00
Carl Lerche 08c21e7bac Fix race condition related bugs (#243)
* Fix races.

This mostly pulls in changes from rust-lang-nursery/futures-rs#881, but
also updates Registration to be a bit more obvious as to what is going
on.

* Reduce spurious wakeups caused by Reactor

This patch adds an ABA guard on token values before registering them
with Mio. This allows catching token reuse and avoid the notification.

This is needed for OS X as the notification is used to determine that a
TCP connect has completed. A spurious notification can potentially cause
write failures.
2018-03-22 09:57:40 -07:00
Aaron Turon 8786741ba9 Update to futures 0.2.0-beta (#246) 2018-03-21 14:21:02 -07:00
Roman 494f0dc176 Runtime builder (#234)
* Split runtime module into files
* Add runtime::Builder to set up thread pool.
2018-03-21 11:23:36 -07:00
Cyril Plisko df9025594c Lapsus clavis (#245) 2018-03-21 11:17:16 -07:00
Aaron Turon 7b1306e6c2 Add top-level tests for futures 0.2 integration (#231) 2018-03-15 16:38:12 -07:00
Sean McArthur b7f4e337be set Runtime thread pool name prefix (#232) 2018-03-15 16:37:32 -07:00
Hiroaki Nakamura d1046db735 Fix condition for updating the current date buffer (#230) 2018-03-15 09:29:55 -07:00
Sam Rijs 923a80e098 Move tokio::net module into tokio tcp/udp crates (#224) 2018-03-14 09:38:59 -07:00
Carl Lerche 64435f5b35 Travis: Move before_deploy out of matrix block (#229) 2018-03-13 15:58:53 -07:00
Aaron Turon d304791c0e Simultaneous futures compat (#172)
This patch adds opt-in support for futures 0.2.
2018-03-13 13:57:35 -07:00
Carl Lerche 5846b3fc2a Reduce AtomicTask spurious notifications (#227) 2018-03-13 13:28:41 -07:00
Carl Lerche 8eb3e58b7d Shutdown the runtime on drop (#214)
Currently, the runtime does not shutdown if the runtime handle is
dropped. This can happen during a panic or when the value is simply
dropped.

This patch forces the runtime to shutdown if it is not explicitly
shutdown.

Fixes #209
2018-03-13 13:14:51 -07:00
Carl Lerche c0a2cc1f9e Add LICENSE file to all sub crates (#226)
* Add LICENSE file to all sub crates
* Update links in README
2018-03-13 13:14:28 -07:00
Carl Lerche 61b2889881 Try compiling Tokio on FreeBSD (#228) 2018-03-13 13:14:12 -07:00
hcpl 5dab821b29 Fix docs markup (#225) 2018-03-13 11:43:05 -07:00
Wesley Moore 2abeff01a5 Fix build on FreeBSD (#218) 2018-03-13 11:24:17 -07:00
Carl Lerche 96a542451d Handle futures that panic on a threadpool (#216)
If a future panics from within the context of a thread pool, the pool
should not be impacted. To do this, polling the future is wrapped with a
catch_unwind. Extra care is taken to ensure that `thread::panicking()`
is set from within the future's drop handle.

Fixes #209
2018-03-13 09:44:14 -07:00
Gray Olson 95899e007d Update comment in udp-codec example (#222) 2018-03-12 10:39:32 -07:00
Jeehoon Kang e6e3c49e0e Remove uses of futures_cpupool (#220) 2018-03-11 11:37:10 -07:00
Carl Lerche 4d514b7eb3 Relicense Tokio exclusively under the MIT license. (#215)
This patch relicenses the Tokio project exclusively under the MIT
license. Before this, the project was dual licensed under MIT and Apache
2. As such, switching to only MIT is permitted.

Fixes #202
2018-03-09 20:07:09 -08:00
Tosil Velkov 2a1585157e Fix wrong file link in examples readme.md (#208) 2018-03-09 12:26:09 -08:00
Igor Gnatenko 189d6baac4 tokio-threadpool: bump rand to 0.4 (#205) 2018-03-09 12:25:38 -08:00
Igor Gnatenko 3ad27e99ec tokio-reactor: bump mio to 0.6.14 (#204)
With 0.6.13 it doesn't compile:
no method named `as_usize` found for type `mio::Ready` in the current scope
2018-03-09 12:25:16 -08:00
218 changed files with 18309 additions and 4472 deletions
+5 -2
View File
@@ -1,11 +1,14 @@
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host x86_64-pc-windows-msvc
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- if NOT "%TARGET%" == "x86_64-pc-windows-msvc" rustup target add %TARGET%
- rustc -V
- cargo -V
+58 -3
View File
@@ -1,23 +1,78 @@
---
language: rust
sudo: false
cache:
- apt
- cargo
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
- rust: 1.21.0
- rust: stable
before_deploy: cargo doc --all --no-deps
- os: osx
- rust: beta
- rust: nightly
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
script:
- |
set -e
if [[ "$TRAVIS_RUST_VERSION" == nightly ]]
then
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests
fi
- cargo test --all
- |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --target $TARGET
cargo check --tests --all --target $TARGET
else
cargo test --all
# Disable these tests for now as they are buggy
#
# cargo test --features unstable-futures
# cargo test --manifest-path tokio-threadpool/Cargo.toml --features unstable-futures
# cargo test --manifest-path tokio-reactor/Cargo.toml --features unstable-futures
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
+27
View File
@@ -1,3 +1,30 @@
# 0.1.7 (June 6, 2018)
* Add `Runtime::block_on` for concurrent runtime (#391).
* Provide handle to `current_thread::Runtime` that allows spawning tasks from
other threads (#340).
* Provide `clock::now()`, a configurable source of time (#381).
# 0.1.6 (May 2, 2018)
* Add asynchronous filesystem APIs (#323).
* Add "current thread" runtime variant (#308).
* `CurrentThread`: Expose inner `Park` instance.
* Improve fairness of `CurrentThread` executor (#313).
# 0.1.5 (March 30, 2018)
* Provide timer API (#266)
# 0.1.4 (March 22, 2018)
* Fix build on FreeBSD (#218)
* Shutdown the Runtime when the handle is dropped (#214)
* Set Runtime thread name prefix for worker threads (#232)
* Add builder for Runtime (#234)
* Extract TCP and UDP types into separate crates (#224)
* Optionally support futures 0.2.
# 0.1.3 (March 09, 2018)
* Fix `CurrentThread::turn` to block on idle (#212).
+25 -14
View File
@@ -5,9 +5,9 @@ name = "tokio"
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.3"
version = "0.1.7"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT/Apache-2.0"
license = "MIT"
readme = "README.md"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
@@ -23,29 +23,43 @@ keywords = ["io", "async", "non-blocking", "futures"]
members = [
"./",
"tokio-codec",
"tokio-current-thread",
"tokio-executor",
"tokio-fs",
"tokio-io",
"tokio-reactor",
"tokio-threadpool",
"tokio-timer",
"tokio-tcp",
"tokio-udp",
"tokio-uds",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
tokio-current-thread = { version = "0.1.0", path = "tokio-current-thread" }
tokio-io = { version = "0.1.6", path = "tokio-io" }
tokio-executor = { version = "0.1.0", path = "tokio-executor" }
tokio-reactor = { version = "0.1.0", path = "tokio-reactor" }
tokio-threadpool = { version = "0.1.0", path = "tokio-threadpool" }
bytes = "0.4"
log = "0.4"
tokio-executor = { version = "0.1.2", path = "tokio-executor" }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor" }
tokio-threadpool = { version = "0.1.4", path = "tokio-threadpool" }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp" }
tokio-udp = { version = "0.1.0", path = "tokio-udp" }
tokio-timer = { version = "0.2.4", path = "tokio-timer" }
tokio-fs = { version = "0.1.0", path = "tokio-fs" }
futures = "0.1.20"
# Needed until `reactor` is removed from `tokio`.
mio = "0.6.14"
slab = "0.4"
iovec = "0.1"
futures = "0.1.18"
[dev-dependencies]
tokio-codec = { version = "0.1.0", path = "tokio-codec" }
bytes = "0.4"
env_logger = { version = "0.4", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
@@ -57,6 +71,3 @@ serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
tokio-io = { path = "tokio-io" }
View File
-201
View File
@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+31 -11
View File
@@ -16,6 +16,7 @@ the Rust programming language. It is:
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
@@ -25,10 +26,13 @@ the Rust programming language. It is:
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio)
[API Docs](https://docs.rs/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
@@ -107,34 +111,50 @@ have greater guarantees of stability.
The crates included as part of Tokio are:
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-current-thread`]: Schedule the execution of futures on the current
thread.
* [`tokio-executor`]: Task execution related traits and utilities.
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
* [`tokio-tcp`]: TCP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-threadpool`]: Schedules the execution of futures across a pool of
threads.
* [ `tokio-timer`]: Time related APIs.
* [`tokio-udp`]: UDP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-uds`]: Unix Domain Socket bindings for use with `tokio-io` and
`tokio-reactor`.
[`tokio-codec`]: tokio-codec
[`tokio-current-thread`]: tokio-current-thread
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-io`]: tokio-io
[`tokio-reactor`]: tokio-reactor
[`tokio-tcp`]: tokio-tcp
[`tokio-threadpool`]: tokio-threadpool
[`tokio-timer`]: tokio-timer
[`tokio-udp`]: tokio-udp
[`tokio-uds`]: tokio-uds
## License
This project is licensed under either of
* Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](LICENSE-MIT) or
http://opensource.org/licenses/MIT)
at your option.
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in tokio by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-1
View File
@@ -13,7 +13,6 @@ mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::executor::current_thread;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
+33
View File
@@ -0,0 +1,33 @@
# TSAN suppressions file for Tokio
# TSAN does not understand fences and `Arc::drop` is implemented using a fence.
# This causes many false positives.
race:Arc*drop
race:arc*Weak*drop
# `std` mpsc is not used in any Tokio code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
# Probably more fences in std.
race:__call_tls_dtors
# The crossbeam deque uses fences.
race:crossbeam_deque
# This is excluded as this race shows up due to using the stealing features of
# the deque. Unfortunately, the implementation uses a fence, which makes tsan
# unhappy.
#
# TODO: It would be nice to not have to filter this out.
race:try_steal_task
# This filters out expected data race in the treiber stack implementations.
# Treiber stacks are inherently racy. The pop operation will attempt to access
# the "next" pointer on the node it is attempting to pop. However, at this
# point it has not gained ownership of the node and another thread might beat
# it and take ownership of the node first (touching the next pointer). The
# original pop operation will fail due to the ABA guard, but tsan still picks
# up the access on the next pointer.
race:Backup::next_sleeper
race:WorkerEntry::set_next_sleeper
+10 -2
View File
@@ -19,6 +19,10 @@ A high level description of each example is:
connections and then echos back any contents that are read from each connected
client.
* [`print_each_packet`](print_each_packet.rs) - this server will create a TCP
listener, accept connections in a loop, and put down in the stdout everything
that's read off of each TCP connection.
* [`echo-udp`](echo-udp.rs) - again your standard "echo server", except for UDP
instead of TCP. This will echo back any packets received to the original
sender.
@@ -34,7 +38,7 @@ A high level description of each example is:
in multiple terminals and use it to chat between the terminals.
* [`chat-combinator`](chat-combinator.rs) - Similar to `chat`, but this uses a
much more functional programming approch using combinators.
much more functional programming approach using combinators.
* [`proxy`](proxy.rs) - an example proxy server that will forward all connected
TCP clients to the remote address specified when starting the program.
@@ -44,9 +48,13 @@ A high level description of each example is:
spawning tasks, and finally framing a TCP connection to discrete
request/response objects.
* [`tinydb`](tinyhttp.rs) - an in-memory database which shows sharing state
* [`tinydb`](tinydb.rs) - an in-memory database which shows sharing state
between all connected clients, notably the key/value store of this database.
* [`udp-client`](udp-client.rs) - a simple `send_dgram`/`recv_dgram` example.
* [`manual-runtime`](manual-runtime.rs) - manually composing a runtime.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
+6 -6
View File
@@ -4,7 +4,7 @@
//! illustrate more concepts.
//!
//! A chat server for telnet clients. After a telnet client connects, the first
//! line should contain the client's name. After that, all lines send by a
//! line should contain the client's name. After that, all lines sent by a
//! client are broadcasted to all other connected clients.
//!
//! Because the client is telnet, lines are delimited by "\r\n".
@@ -157,7 +157,7 @@ impl Peer {
/// This is where a connected client is managed.
///
/// A `Peer` is also a future representing completly processing the client.
/// A `Peer` is also a future representing completely processing the client.
///
/// When a `Peer` is created, the first line (representing the client's name)
/// has already been read. When the socket closes, the `Peer` future completes.
@@ -216,9 +216,9 @@ impl Future for Peer {
if let Some(message) = line {
// Append the peer's name to the front of the line:
let mut line = self.name.clone();
line.put(": ");
line.put(&message);
line.put("\r\n");
line.extend_from_slice(b": ");
line.extend_from_slice(&message);
line.extend_from_slice(b"\r\n");
// We're using `Bytes`, which allows zero-copy clones (by
// storing the data in an Arc internally).
@@ -290,7 +290,7 @@ impl Lines {
fn poll_flush(&mut self) -> Poll<(), io::Error> {
// As long as there is buffered data to write, try to write it.
while !self.wr.is_empty() {
// Try to read some bytes from the socket
// Try to write some bytes to the socket
let n = try_ready!(self.socket.poll_write(&self.wr));
// As long as the wr is not empty, a successful write should
+13 -6
View File
@@ -17,6 +17,7 @@
#![deny(warnings)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
@@ -82,7 +83,7 @@ fn main() {
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio_io::codec::{Encoder, Decoder};
use tokio_codec::{Encoder, Decoder};
/// A simple `Codec` implementation that just ships bytes around.
///
@@ -120,6 +121,7 @@ mod codec {
mod tcp {
use tokio;
use tokio_codec::Decoder;
use tokio::net::TcpStream;
use tokio::prelude::*;
@@ -151,7 +153,7 @@ mod tcp {
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
Box::new(tcp.map(move |stream| {
let (sink, stream) = stream.framed(Bytes).split();
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
@@ -198,24 +200,29 @@ mod udp {
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
tokio::spawn(stdin.map(move |chunk| {
let forward_stdin = stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
Box::new(stream.filter_map(move |(chunk, src)| {
let receive = stream.filter_map(move |(chunk, src)| {
if src == addr {
Some(chunk.into())
} else {
None
}
}))
});
Box::new(future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
}).flatten_stream())
}
}
+1 -1
View File
@@ -68,6 +68,6 @@ fn main() {
// `map_err` handles the error by logging it and maps the future to a type
// that can be spawned.
//
// `tokio::run` spanws the task on the Tokio runtime and starts running.
// `tokio::run` spawns the task on the Tokio runtime and starts running.
tokio::run(server.map_err(|e| println!("server error = {:?}", e)));
}
+1 -1
View File
@@ -3,7 +3,7 @@
//! This server will create a TCP listener, accept connections in a loop, and
//! write back everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread poool, each TCP connection is
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
+86
View File
@@ -0,0 +1,86 @@
//! An example how to manually assemble a runtime and run some tasks on it.
//!
//! This is closer to the single-threaded runtime than the default tokio one, as it is simpler to
//! grasp. There are conceptually similar, but the multi-threaded one would be more code. If you
//! just want to *use* a single-threaded runtime, use the one provided by tokio directly
//! (`tokio::runtime::current_thread::Runtime::new()`. This is a demonstration only.
//!
//! Note that the error handling is a bit left out. Also, the `run` could be modified to return the
//! result of the provided future.
extern crate futures;
extern crate tokio;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_timer;
use std::io::Error as IoError;
use std::time::{Duration, Instant};
use futures::{future, Future};
use tokio_current_thread::CurrentThread;
use tokio_reactor::Reactor;
use tokio_timer::timer::{self, Timer};
/// Creates a "runtime".
///
/// This is similar to running `tokio::runtime::current_thread::Runtime::new()`.
fn run<F: Future<Item = (), Error = ()>>(f: F) -> Result<(), IoError> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new(reactor);
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor generate some new stimuli for the
// futures to continue in their life.
let mut executor = CurrentThread::new_with_park(timer);
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the current single-threaded
// executor when used. This is a trick, because we need two mutable references to the
// executor (one to run the provided future, another to install as the default one). We
// use the fake one here as the default one.
let mut default_executor = tokio_current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
// Run the provided future
executor.block_on(f).unwrap();
// Run all the other futures that are still left in the executor
executor.run().unwrap();
});
});
});
Ok(())
}
fn main() {
run(future::lazy(|| {
// Here comes the application logic. It can spawn further tasks by tokio_current_thread::spawn().
// It also can use the default reactor and create timeouts.
// Connect somewhere. And then do nothing with it. Yes, useless.
//
// This will use the default reactor which runs in the current thread.
let connect = tokio::net::TcpStream::connect(&"127.0.0.1:53".parse().unwrap())
.map(|_| println!("Connected"))
.map_err(|e| println!("Failed to connect: {}", e));
// We can spawn it without requiring Send. This would panic if we run it outside of the
// `run` (or outside of anything else)
tokio_current_thread::spawn(connect);
// We can also create timeouts.
let deadline = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(5))
.map(|()| println!("5 seconds are over"))
.map_err(|e| println!("Failed to wait: {}", e));
// We can spawn on the default executor, which is also the local one.
tokio::executor::spawn(deadline);
Ok(())
})).unwrap();
}
+149
View File
@@ -0,0 +1,149 @@
//! A "print-each-packet" server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! put down in the stdout everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example print\_each\_packet
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be written to terminal!
//!
//! Minimal js example:
//!
//! ```js
//! var net = require("net");
//!
//! var listenPort = 8080;
//!
//! var server = net.createServer(function (socket) {
//! socket.on("data", function (bytes) {
//! console.log("bytes", bytes);
//! });
//!
//! socket.on("end", function() {
//! console.log("Socket received FIN packet and closed connection");
//! });
//! socket.on("error", function (error) {
//! console.log("Socket closed with error", error);
//! });
//!
//! socket.on("close", function (with_error) {
//! if (with_error) {
//! console.log("Socket closed with result: Err(SomeError)");
//! } else {
//! console.log("Socket closed with result: Ok(())");
//! }
//! });
//!
//! });
//!
//! server.listen(listenPort);
//!
//! console.log("Listening on:", listenPort);
//! ```
//!
#![deny(warnings)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_codec::{Decoder, BytesCodec};
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
fn main() {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We're parsing each socket with the `BytesCodec` included in `tokio_io`,
// and then we `split` each codec into the reader/writer halves.
//
// See https://docs.rs/tokio-codec/0.1/src/tokio_codec/bytes_codec.rs.html
let framed = BytesCodec::new().framed(socket);
let (_writer, reader) = framed.split();
let processor = reader
.for_each(|bytes| {
println!("bytes: {:?}", bytes);
Ok(())
})
// After our copy operation is complete we just print out some helpful
// information.
.and_then(|()| {
println!("Socket received FIN packet and closed connection");
Ok(())
})
.or_else(|err| {
println!("Socket closed with error: {:?}", err);
// We have to return the error to catch it in the next ``.then` call
Err(err)
})
.then(|result| {
println!("Socket closed with result: {:?}", result);
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(processor)
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
}
+1 -1
View File
@@ -1,7 +1,7 @@
//! A proxy that forwards data to another server and forwards that server's
//! responses back to clients.
//!
//! Because the Tokio runtime uses a thread poool, each TCP connection is
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
+5 -4
View File
@@ -21,6 +21,7 @@ extern crate serde_derive;
extern crate serde_json;
extern crate time;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
use std::{env, fmt, io};
@@ -29,7 +30,7 @@ use std::net::SocketAddr;
use tokio::net::{TcpStream, TcpListener};
use tokio::prelude::*;
use tokio_io::codec::{Encoder, Decoder};
use tokio_codec::{Encoder, Decoder};
use bytes::BytesMut;
use http::header::HeaderValue;
@@ -55,10 +56,10 @@ fn main() {
}
fn process(socket: TcpStream) {
let (tx, rx) = socket
let (tx, rx) =
// Frame the socket using the `Http` protocol. This maps the TCP socket
// to a Stream + Sink of HTTP frames.
.framed(Http)
Http.framed(socket)
// This splits a single `Stream + Sink` value into two separate handles
// that can be used independently (even on different tasks or threads).
.split();
@@ -274,7 +275,7 @@ mod date {
LAST.with(|cache| {
let mut cache = cache.borrow_mut();
let now = time::get_time();
if now > cache.next_update {
if now >= cache.next_update {
cache.update(now);
}
f.write_str(cache.buffer())
+74
View File
@@ -0,0 +1,74 @@
//! A UDP client that just sends everything it gets via `stdio` in a single datagram, and then
//! waits for a reply.
//!
//! For the reasons of simplicity data from `stdio` is read until `EOF` in a blocking manner.
//!
//! You can test this out by running an echo server:
//!
//! ```
//! $ cargo run --example echo-udp -- 127.0.0.1:8080
//! ```
//!
//! and running the client in another terminal:
//!
//! ```
//! $ cargo run --example udp-client
//! ```
//!
//! You can optionally provide any custom endpoint address for the client:
//!
//! ```
//! $ cargo run --example udp-client -- 127.0.0.1:8080
//! ```
//!
//! Don't forget to pass `EOF` to the standard input of the client!
//!
//! Please mind that since the UDP protocol doesn't have any capabilities to detect a broken
//! connection the server needs to be run first, otherwise the client will block forever.
extern crate futures;
extern crate tokio;
use std::env;
use std::io::stdin;
use std::net::SocketAddr;
use tokio::net::UdpSocket;
use tokio::prelude::*;
fn get_stdin_data() -> Vec<u8> {
let mut buf = Vec::new();
stdin().read_to_end(&mut buf).unwrap();
buf
}
fn main() {
let remote_addr: SocketAddr = env::args()
.nth(1)
.unwrap_or("127.0.0.1:8080".into())
.parse()
.unwrap();
// We use port 0 to let the operating system allocate an available port for us.
let local_addr: SocketAddr = if remote_addr.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
}.parse()
.unwrap();
let socket = UdpSocket::bind(&local_addr).unwrap();
const MAX_DATAGRAM_SIZE: usize = 65_507;
let processing = socket
.send_dgram(get_stdin_data(), &remote_addr)
.and_then(|(socket, _)| socket.recv_dgram(vec![0u8; MAX_DATAGRAM_SIZE]))
.map(|(_, data, len, _)| {
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
)
})
.wait();
match processing {
Ok(_) => {}
Err(e) => eprintln!("Encountered an error: {}", e),
}
}
+3 -2
View File
@@ -9,6 +9,7 @@
#![deny(warnings)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate env_logger;
@@ -16,7 +17,7 @@ use std::net::SocketAddr;
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_io::codec::BytesCodec;
use tokio_codec::BytesCodec;
fn main() {
let _ = env_logger::init();
@@ -28,7 +29,7 @@ fn main() {
let b = UdpSocket::bind(&addr).unwrap();
let b_addr = b.local_addr().unwrap();
// We're parsing each socket with the `LineCodec` defined above, and then we
// We're parsing each socket with the `BytesCodec` included in `tokio_io`, and then we
// `split` each codec into the sink/stream halves.
let (a_sink, a_stream) = UdpFramed::new(a, BytesCodec::new()).split();
let (b_sink, b_stream) = UdpFramed::new(b, BytesCodec::new()).split();
+15
View File
@@ -0,0 +1,15 @@
//! A configurable source of time.
//!
//! This module provides the [`now`][n] function, which returns an `Instant`
//! representing "now". The source of time used by this function is configurable
//! (via the [`tokio-timer`] crate) and allows mocking out the source of time in
//! tests or performing caching operations to reduce the number of syscalls.
//!
//! Note that, because the source of time is configurable, it is possible to
//! observe non-monotonic behavior when calling [`now`][n] from different
//! executors.
//!
//! [n]: fn.now.html
//! [`tokio-timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/clock/index.html
pub use tokio_timer::clock::now;
+23 -566
View File
@@ -1,3 +1,5 @@
#![allow(deprecated)]
//! Execute many tasks concurrently on the current thread.
//!
//! [`CurrentThread`] is an executor that keeps tasks on the same thread that
@@ -102,57 +104,24 @@
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
#![allow(deprecated)]
pub use tokio_current_thread::{
BlockError,
CurrentThread,
Entered,
Handle,
RunError,
RunTimeoutError,
TaskExecutor,
Turn,
TurnError,
block_on_all,
spawn,
};
mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::{self, Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::{executor, Async, Future};
use futures::future::{self, Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::marker::PhantomData;
use std::rc::Rc;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
/// Execute futures and receive unpark notifications.
scheduler: Scheduler<P::Unpark>,
/// Current number of futures being executed
num_futures: usize,
/// Thread park handle
park: P,
}
/// Executes futures on the current thread.
///
/// All futures executed using this executor will be executed on the current
/// thread. As such, `run` will wait for these futures to complete before
/// returning.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
// Prevent the handle from moving across threads.
_p: ::std::marker::PhantomData<Rc<()>>,
}
/// Returned by the `turn` function
#[derive(Debug)]
pub struct Turn(());
/// A `CurrentThread` instance bound to a supplied execution conext.
pub struct Entered<'a, P: Park + 'a> {
executor: &'a mut CurrentThread<P>,
enter: &'a mut Enter,
}
use futures::future::{self};
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
@@ -162,54 +131,17 @@ pub struct Context<'a> {
_p: PhantomData<&'a ()>,
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
_p: (),
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
/// Error returned by the `run_timeout` function.
#[derive(Debug)]
pub struct RunTimeoutError {
timeout: bool,
}
/// Error returned by the `turn` function.
#[derive(Debug)]
pub struct TurnError {
_p: (),
}
/// Error returned by the `block_on` function.
#[derive(Debug)]
pub struct BlockError<T> {
inner: Option<T>,
}
/// This is mostly split out to make the borrow checker happy.
struct Borrow<'a, U: 'a> {
scheduler: &'a mut Scheduler<U>,
num_futures: &'a mut usize,
}
trait SpawnLocal {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>);
}
struct CurrentRunner {
spawn: Cell<Option<*mut SpawnLocal>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
});
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[allow(deprecated)]
pub fn run<F, R>(f: F) -> R
where F: FnOnce(&mut Context) -> R
where F: FnOnce(&mut Context) -> R
{
let mut context = Context {
cancel: Cell::new(false),
@@ -230,484 +162,9 @@ where F: FnOnce(&mut Context) -> R
ret
}
/// Run the executor bootstrapping the execution with the provided future.
///
/// This creates a new [`CurrentThread`] executor, spawns the provided future,
/// and blocks the current thread until the provided future and **all**
/// subsequently spawned futures complete. In other words:
///
/// * If the provided boostrap future does **not** spawn any additional tasks,
/// `block_on_all` returns once `future` completes.
/// * If the provided bootstrap future **does** spawn additional tasks, then
/// `block_on_all` returns once **all** spawned futures complete.
///
/// See [module level][mod] documentation for more details.
///
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
{
let mut current_thread = CurrentThread::new();
let ret = current_thread.block_on(future);
current_thread.run().unwrap();
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
}
/// Executes a future on the current thread.
///
/// The provided future must complete or be canceled before `run` will return.
///
/// Unlike [`tokio::spawn`], this function will always spawn on a
/// `CurrentThread` executor and is able to spawn futures that are not `Send`.
///
/// # Panics
///
/// This function can only be invoked from the context of a `run` call; any
/// other use will result in a panic.
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
{
TaskExecutor::current()
.spawn_local(Box::new(future))
.unwrap();
}
// ===== impl CurrentThread =====
impl CurrentThread<ParkThread> {
/// Create a new instance of `CurrentThread`.
pub fn new() -> Self {
CurrentThread::new_with_park(ParkThread::new())
}
}
impl<P: Park> CurrentThread<P> {
/// Create a new instance of `CurrentThread` backed by the given park
/// handle.
pub fn new_with_park(park: P) -> Self {
let unpark = park.unpark();
CurrentThread {
scheduler: Scheduler::new(unpark),
num_futures: 0,
park,
}
}
/// Returns `true` if the executor is currently idle.
///
/// An idle executor is defined by not currently having any spawned tasks.
pub fn is_idle(&self) -> bool {
self.num_futures == 0
}
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).turn(duration)
}
/// Bind `CurrentThread` instance with an execution context.
pub fn enter<'a>(&'a mut self, enter: &'a mut Enter) -> Entered<'a, P> {
Entered {
executor: self,
enter,
}
}
fn borrow(&mut self) -> Borrow<P::Unpark> {
Borrow {
scheduler: &mut self.scheduler,
num_futures: &mut self.num_futures,
}
}
}
impl tokio_executor::Executor for CurrentThread {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.borrow().spawn_local(future);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures)
.finish()
}
}
// ===== impl Entered =====
impl<'a, P: Park> Entered<'a, P> {
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
match res {
Ok(Async::Ready(e)) => return Ok(e),
Err(e) => return Err(BlockError { inner: Some(e) }),
Ok(Async::NotReady) => {}
}
self.tick();
if let Err(_) = self.executor.park.park() {
return Err(BlockError { inner: None });
}
}
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
if !self.tick() {
let res = match duration {
Some(duration) => self.executor.park.park_timeout(duration),
None => self.executor.park.park(),
};
if res.is_err() {
return Err(TurnError { _p: () });
}
self.tick();
}
Ok(Turn(()))
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
if self.executor.is_idle() {
// Nothing to do
return Ok(());
}
let mut time = dur.map(|dur| (Instant::now() + dur, dur));
loop {
self.tick();
if self.executor.is_idle() {
return Ok(());
}
match time {
Some((until, rem)) => {
if let Err(_) = self.executor.park.park_timeout(rem) {
return Err(RunTimeoutError::new(false));
}
let now = Instant::now();
if now >= until {
return Err(RunTimeoutError::new(true));
}
time = Some((until, until - now));
}
None => {
if let Err(_) = self.executor.park.park() {
return Err(RunTimeoutError::new(false));
}
}
}
}
}
/// Returns `true` if any futures were processed
fn tick(&mut self) -> bool {
self.executor.scheduler.tick(
&mut *self.enter,
&mut self.executor.num_futures)
}
}
impl<'a, P: Park> fmt::Debug for Entered<'a, P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Entered")
.field("executor", &self.executor)
.field("enter", &self.enter)
.finish()
}
}
// ===== impl TaskExecutor =====
#[deprecated(since = "0.1.2", note = "use TaskExecutor::current instead")]
#[doc(hidden)]
pub fn task_executor() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
TaskExecutor::current()
}
impl TaskExecutor {
/// Returns an executor that executes futures on the current thread.
///
/// The user of `TaskExecutor` must ensure that when a future is submitted,
/// that it is done within the context of a call to `run`.
///
/// For more details, see the [module level](index.html) documentation.
pub fn current() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
}
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.spawn_local(future)
}
fn status(&self) -> Result<(), SpawnError> {
CURRENT.with(|current| {
if current.spawn.get().is_some() {
Ok(())
} else {
Err(SpawnError::shutdown())
}
})
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future)) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
}
})
}
}
// ===== impl Context =====
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
// ===== impl Borrow =====
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.set_spawn(self, || {
f()
})
})
}
}
impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>) {
*self.num_futures += 1;
self.scheduler.schedule(future);
}
}
// ===== impl CurrentRunner =====
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
{
struct Reset<'a>(&'a CurrentRunner);
impl<'a> Drop for Reset<'a> {
fn drop(&mut self) {
self.0.spawn.set(None);
}
}
let _reset = Reset(self);
let spawn = unsafe { hide_lt(spawn as *mut SpawnLocal) };
self.spawn.set(Some(spawn));
f()
}
}
unsafe fn hide_lt<'a>(p: *mut (SpawnLocal + 'a)) -> *mut (SpawnLocal + 'static) {
use std::mem;
mem::transmute(p)
}
// ===== impl RunTimeoutError =====
impl RunTimeoutError {
fn new(timeout: bool) -> Self {
RunTimeoutError { timeout }
}
/// Returns `true` if the error was caused by the operation timeing out.
pub fn is_timeout(&self) -> bool {
self.timeout
}
}
impl From<tokio_executor::EnterError> for RunTimeoutError {
fn from(_: tokio_executor::EnterError) -> Self {
RunTimeoutError::new(false)
}
}
// ===== impl BlockError =====
impl<T> BlockError<T> {
/// Returns the error yielded by the future being blocked on
pub fn into_inner(self) -> Option<T> {
self.inner
}
}
impl<T> From<tokio_executor::EnterError> for BlockError<T> {
fn from(_: tokio_executor::EnterError) -> Self {
BlockError { inner: None }
}
}
+38 -22
View File
@@ -5,7 +5,7 @@
//! the future must be submitted to an executor. A future that is submitted to
//! an executor is called a "task".
//!
//! The executor executor is responsible for ensuring that [`Future::poll`] is
//! The executor is responsible for ensuring that [`Future::poll`] is
//! called whenever the task is [notified]. Notification happens when the
//! internal state of a task transitions from "not ready" to ready. For
//! example, a socket might have received data and a call to `read` will now be
@@ -13,16 +13,8 @@
//!
//! The specific strategy used to manage the tasks is left up to the
//! executor. There are two main flavors of executors: single-threaded and
//! multithreaded. This module provides both.
//!
//! * **[`current_thread`]**: A single-threaded executor that support spawning
//! tasks that are not `Send`. It guarantees that tasks will be executed on
//! the same thread from which they are spawned.
//!
//! * **[`thread_pool`]**: A multi-threaded executor that maintains a pool of
//! threads. Tasks are spawned to one of the threads in the pool and executed.
//! The pool employes a [work-stealing] strategy for optimizing how tasks get
//! spread across the available threads.
//! multithreaded. Tokio provides implementation for both of these in the
//! [`runtime`] module.
//!
//! # `Executor` trait.
//!
@@ -36,22 +28,23 @@
//! executor. This value will often be set to the executor itself, but it is
//! possible that the default executor might be set to a different executor.
//!
//! For example, the [`current_thread`] executor might set the default executor
//! to a thread pool instead of itself, allowing futures to spawn new tasks onto
//! the thread pool when those tasks are `Send`.
//! For example, a single threaded executor might set the default executor to a
//! thread pool instead of itself, allowing futures to spawn new tasks onto the
//! thread pool when those tasks are `Send`.
//!
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
//! [notified]: https://docs.rs/futures/0.1/futures/executor/trait.Notify.html#tymethod.notify
//! [`current_thread`]: current_thread/index.html
//! [`thread_pool`]: thread_pool/index.html
//! [work-stealing]: https://en.wikipedia.org/wiki/Work_stealing
//! [`tokio-executor`]: #
//! [`Executor`]: #
//! [`spawn`]: #
//! [`runtime`]: ../runtime/index.html
//! [`tokio-executor`]: https://docs.rs/tokio-executor/0.1
//! [`Executor`]: trait.Executor.html
//! [`spawn`]: fn.spawn.html
#[deprecated(since = "0.1.8", note = "use tokio-current-thread crate instead")]
#[doc(hidden)]
pub mod current_thread;
#[deprecated(since = "0.1.8", note = "use tokio-threadpool crate instead")]
#[doc(hidden)]
pub mod thread_pool {
//! Maintains a pool of threads across which the set of spawned tasks are
//! executed.
@@ -137,6 +130,9 @@ pub use tokio_executor::{Executor, DefaultExecutor, SpawnError};
use futures::{Future, IntoFuture};
use futures::future::{self, FutureResult};
#[cfg(feature = "unstable-futures")]
use futures2;
/// Return value from the `spawn` function.
///
/// Currently this value doesn't actually provide any functionality. However, it
@@ -198,7 +194,7 @@ pub struct Spawn(());
/// onto the default executor returns an error. To avoid the panic, use
/// [`DefaultExecutor`].
///
/// [`DefaultExecutor`]: #
/// [`DefaultExecutor`]: struct.DefaultExecutor.html
pub fn spawn<F>(f: F) -> Spawn
where F: Future<Item = (), Error = ()> + 'static + Send
{
@@ -206,6 +202,15 @@ where F: Future<Item = (), Error = ()> + 'static + Send
Spawn(())
}
/// Like `spawn`, but compatible with futures 0.2
#[cfg(feature = "unstable-futures")]
pub fn spawn2<F>(f: F) -> Spawn
where F: futures2::Future<Item = (), Error = futures2::Never> + 'static + Send
{
::tokio_executor::spawn2(f);
Spawn(())
}
impl IntoFuture for Spawn {
type Future = FutureResult<(), ()>;
type Item = ();
@@ -215,3 +220,14 @@ impl IntoFuture for Spawn {
future::ok(())
}
}
#[cfg(feature = "unstable-futures")]
impl futures2::IntoFuture for Spawn {
type Future = futures2::future::FutureResult<(), ()>;
type Item = ();
type Error = ();
fn into_future(self) -> Self::Future {
futures2::future::ok(())
}
}
+14
View File
@@ -0,0 +1,14 @@
//! Asynchronous filesystem manipulation operations.
//!
//! This module contains basic methods and types for manipulating the contents
//! of the local filesystem from within the context of the Tokio runtime.
//!
//! Unlike *most* other Tokio APIs, the filesystem APIs **must** be used from
//! the context of the Tokio runtime as they require Tokio specific features to
//! function.
pub use tokio_fs::{
file,
File,
OpenOptions,
};
+55 -8
View File
@@ -8,9 +8,11 @@
//! * A [reactor][reactor] backed by the operating system's event queue (epoll, kqueue,
//! IOCP, etc...).
//! * Asynchronous [TCP and UDP][net] sockets.
//! * Asynchronous [filesystem][fs] operations.
//! * [Timer][timer] API for scheduling work in the future.
//!
//! Tokio is built using futures (provided by the [futures] crate) as the
//! abstraction for managing the complexity of asynchronous programming.
//! Tokio is built using [futures] as the abstraction for managing the
//! complexity of asynchronous programming.
//!
//! Guide level documentation is found on the [website].
//!
@@ -62,29 +64,38 @@
//! }
//! ```
#![doc(html_root_url = "https://docs.rs/tokio/0.1.3")]
#![doc(html_root_url = "https://docs.rs/tokio/0.1.5")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
extern crate bytes;
#[macro_use]
extern crate futures;
extern crate iovec;
extern crate mio;
extern crate slab;
extern crate tokio_current_thread;
extern crate tokio_io;
extern crate tokio_executor;
extern crate tokio_fs;
extern crate tokio_reactor;
extern crate tokio_threadpool;
extern crate tokio_timer;
extern crate tokio_tcp;
extern crate tokio_udp;
#[macro_use]
extern crate log;
#[cfg(feature = "unstable-futures")]
extern crate futures2;
pub mod clock;
pub mod executor;
pub mod fs;
pub mod net;
pub mod reactor;
pub mod runtime;
pub mod timer;
pub mod util;
pub use executor::spawn;
#[cfg(feature = "unstable-futures")]
pub use executor::spawn2;
pub use runtime::run;
pub mod io {
@@ -94,15 +105,35 @@ pub mod io {
//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which extend the
//! `Read` and `Write` traits of the standard library.
//!
//! # AsyncRead and AsyncWrite
//!
//! [`AsyncRead`] and [`AsyncWrite`] must only be implemented for
//! non-blocking I/O types that integrate with the futures type system. In
//! other words, these types must never block the thread, and instead the
//! current task is notified when the I/O resource is ready.
//!
//! # Standard input and output
//!
//! Tokio provides asynchronous APIs to standard [input], [output], and [error].
//! These APIs are very similar to the ones provided by `std`, but they also
//! implement [`AsyncRead`] and [`AsyncWrite`].
//!
//! Unlike *most* other Tokio APIs, the standard input / output APIs
//! **must** be used from the context of the Tokio runtime as they require
//! Tokio specific features to function.
//!
//! [input]: fn.stdin.html
//! [output]: fn.stdout.html
//! [error]: fn.stderr.html
//!
//! # Utility functions
//!
//! Utilities functions are provided for working with [`AsyncRead`] /
//! [`AsyncWrite`] types. For example, [`copy`] asynchronously copies all
//! data from a source to a destination.
//!
//! # `std` re-exports
//!
//! Additionally, [`Read`], [`Write`], [`Error`], [`ErrorKind`], and
//! [`Result`] are re-exported from `std::io` for ease of use.
//!
@@ -120,6 +151,16 @@ pub mod io {
AsyncWrite,
};
// standard input, output, and error
pub use tokio_fs::{
stdin,
Stdin,
stdout,
Stdout,
stderr,
Stderr,
};
// Utils
pub use tokio_io::io::{
copy,
@@ -134,10 +175,12 @@ pub mod io {
ReadToEnd,
read_until,
ReadUntil,
ReadHalf,
shutdown,
Shutdown,
write_all,
WriteAll,
WriteHalf,
};
// Re-export io::Error so that users don't have to deal
@@ -169,6 +212,10 @@ pub mod prelude {
AsyncWrite,
};
pub use util::{
FutureExt,
};
pub use ::std::io::{
Read,
Write,
+4 -7
View File
@@ -27,7 +27,7 @@
//! Reading and writing to it can be done using futures, which return the
//! [`RecvDgram`] and [`SendDgram`] structs respectively.
//!
//! For convience it's also possible to convert raw datagrams into higher-level
//! For convenience it's also possible to convert raw datagrams into higher-level
//! frames.
//!
//! [`UdpSocket`]: struct.UdpSocket.html
@@ -36,9 +36,6 @@
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`framed`]: struct.UdpSocket.html#method.framed
mod tcp;
mod udp;
pub use self::tcp::{TcpStream, ConnectFuture};
pub use self::tcp::{TcpListener, Incoming};
pub use self::udp::{UdpSocket, UdpFramed, SendDgram, RecvDgram};
pub use tokio_tcp::{TcpStream, ConnectFuture};
pub use tokio_tcp::{TcpListener, Incoming};
pub use tokio_udp::{UdpSocket, UdpFramed, SendDgram, RecvDgram};
-8
View File
@@ -1,8 +0,0 @@
mod incoming;
mod listener;
mod stream;
pub use self::incoming::Incoming;
pub use self::listener::TcpListener;
pub use self::stream::TcpStream;
pub use self::stream::ConnectFuture;
-9
View File
@@ -1,9 +0,0 @@
mod frame;
mod socket;
mod send_dgram;
mod recv_dgram;
pub use self::frame::UdpFramed;
pub use self::socket::UdpSocket;
pub use self::send_dgram::SendDgram;
pub use self::recv_dgram::RecvDgram;
+2 -12
View File
@@ -428,21 +428,11 @@ fn usize2ready(bits: usize) -> Ready {
ready | platform::usize2ready(bits)
}
#[cfg(all(unix, not(target_os = "fuchsia")))]
#[cfg(unix)]
mod platform {
use mio::Ready;
use mio::unix::UnixReady;
#[cfg(target_os = "dragonfly")]
pub fn all() -> Ready {
hup() | UnixReady::aio()
}
#[cfg(target_os = "freebsd")]
pub fn all() -> Ready {
hup() | UnixReady::aio() | UnixReady::lio()
}
const HUP: usize = 1 << 2;
const ERROR: usize = 1 << 3;
const AIO: usize = 1 << 4;
@@ -526,7 +516,7 @@ mod platform {
}
}
#[cfg(any(windows, target_os = "fuchsia"))]
#[cfg(windows)]
mod platform {
use mio::Ready;
+148
View File
@@ -0,0 +1,148 @@
use runtime::{Inner, Runtime};
use reactor::Reactor;
use std::io;
use tokio_reactor;
use tokio_threadpool::Builder as ThreadPoolBuilder;
use tokio_threadpool::park::DefaultPark;
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
/// Builds Tokio Runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate tokio_threadpool;
/// # use tokio::runtime::Builder;
///
/// # pub fn main() {
/// // create and configure ThreadPool
/// let mut threadpool_builder = tokio_threadpool::Builder::new();
/// threadpool_builder
/// .name_prefix("my-runtime-worker-")
/// .pool_size(4);
///
/// // build Runtime
/// let runtime = Builder::new()
/// .threadpool_builder(threadpool_builder)
/// .build();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
#[derive(Debug)]
pub struct Builder {
/// Thread pool specific builder
threadpool_builder: ThreadPoolBuilder,
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
let mut threadpool_builder = ThreadPoolBuilder::new();
threadpool_builder.name_prefix("tokio-runtime-worker-");
Builder {
threadpool_builder,
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Set builder to set up the thread pool instance.
pub fn threadpool_builder(&mut self, val: ThreadPoolBuilder) -> &mut Self {
self.threadpool_builder = val;
self
}
/// Create the configured `Runtime`.
///
/// The returned `ThreadPool` instance is ready to spawn tasks.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::runtime::Builder;
/// # pub fn main() {
/// let runtime = Builder::new().build().unwrap();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
pub fn build(&mut self) -> io::Result<Runtime> {
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
// Get a handle to the clock for the runtime.
let clock1 = self.clock.clone();
let clock2 = clock1.clone();
let timers = Arc::new(Mutex::new(HashMap::<_, timer::Handle>::new()));
let t1 = timers.clone();
// Spawn a reactor on a background thread.
let reactor = Reactor::new()?.background()?;
// Get a handle to the reactor.
let reactor_handle = reactor.handle().clone();
let pool = self.threadpool_builder
.around_worker(move |w, enter| {
let timer_handle = t1.lock().unwrap()
.get(w.id()).unwrap()
.clone();
tokio_reactor::with_default(&reactor_handle, enter, |enter| {
clock::with_default(&clock1, enter, |enter| {
timer::with_default(&timer_handle, enter, |_| {
w.run();
});
})
});
})
.custom_park(move |worker_id| {
// Create a new timer
let timer = Timer::new_with_now(DefaultPark::new(), clock2.clone());
timers.lock().unwrap()
.insert(worker_id.clone(), timer.handle());
timer
})
.build();
Ok(Runtime {
inner: Some(Inner {
reactor,
pool,
}),
})
}
}
+88
View File
@@ -0,0 +1,88 @@
use executor::current_thread::CurrentThread;
use runtime::current_thread::Runtime;
use tokio_reactor::Reactor;
use tokio_timer::clock::Clock;
use tokio_timer::timer::Timer;
use std::io;
/// Builds a Single-threaded runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// extern crate tokio;
/// extern crate tokio_timer;
///
/// use tokio::runtime::current_thread::Builder;
/// use tokio_timer::clock::Clock;
///
/// # pub fn main() {
/// // build Runtime
/// let runtime = Builder::new()
/// .clock(Clock::new())
/// .build();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
#[derive(Debug)]
pub struct Builder {
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
Builder {
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Create the configured `Runtime`.
pub fn build(&mut self) -> io::Result<Runtime> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new_with_now(reactor, self.clock.clone());
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor to generate some new stimuli for the
// futures to continue in their life.
let executor = CurrentThread::new_with_park(timer);
let runtime = Runtime::new2(
reactor_handle,
timer_handle,
self.clock.clone(),
executor);
Ok(runtime)
}
}
+70
View File
@@ -0,0 +1,70 @@
//! A runtime implementation that runs everything on the current thread.
//!
//! [`current_thread::Runtime`][rt] is similar to the primary
//! [`Runtime`][concurrent-rt] except that it runs all components on the current
//! thread instead of using a thread pool. This means that it is able to spawn
//! futures that do not implement `Send`.
//!
//! Same as the default [`Runtime`][concurrent-rt], the
//! [`current_thread::Runtime`][rt] includes:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! Note that [`current_thread::Runtime`][rt] does not implement `Send` itself
//! and cannot be safely moved to other threads.
//!
//! # Spawning from other threads
//!
//! While [`current_thread::Runtime`][rt] does not implement `Send` and cannot
//! safely be moved to other threads, it provides a `Handle` that can be sent
//! to other threads and allows to spawn new tasks from there.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//! use std::thread;
//!
//! # fn main() {
//! let mut runtime = Runtime::new().unwrap();
//! let handle = runtime.handle();
//!
//! thread::spawn(move || {
//! handle.spawn(future::ok(()));
//! }).join().unwrap();
//!
//! # /*
//! runtime.run().unwrap();
//! # */
//! # }
//! ```
//!
//! # Examples
//!
//! Creating a new `Runtime` and running a future `f` until its completion and
//! returning its result.
//!
//! ```
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//!
//! let mut runtime = Runtime::new().unwrap();
//!
//! // Use the runtime...
//! // runtime.block_on(f); // where f is a future
//! ```
//!
//! [rt]: struct.Runtime.html
//! [concurrent-rt]: ../struct.Runtime.html
//! [chan]: https://docs.rs/futures/0.1/futures/sync/mpsc/fn.channel.html
mod builder;
mod runtime;
pub use self::builder::Builder;
pub use self::runtime::{Runtime, Handle};
+185
View File
@@ -0,0 +1,185 @@
use tokio_current_thread::{self as current_thread, CurrentThread};
use tokio_current_thread::Handle as ExecutorHandle;
use runtime::current_thread::Builder;
use tokio_reactor::{self, Reactor};
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
use tokio_executor;
use futures::Future;
use std::io;
/// Single-threaded runtime provides a way to start reactor
/// and executor on the current thread.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
#[derive(Debug)]
pub struct Runtime {
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>,
}
/// Handle to spawn a future on the corresponding `CurrentThread` runtime instance
#[derive(Debug, Clone)]
pub struct Handle(ExecutorHandle);
impl Handle {
/// Spawn a future onto the `CurrentThread` runtime instance corresponding to this handle
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the `CurrentThread`
/// instance of the `Handle` does not exist anymore.
pub fn spawn<F>(&self, future: F) -> Result<(), tokio_executor::SpawnError>
where F: Future<Item = (), Error = ()> + Send + 'static {
self.0.spawn(future)
}
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
inner: current_thread::RunError,
}
impl Runtime {
/// Returns a new runtime initialized with default configuration values.
pub fn new() -> io::Result<Runtime> {
Builder::new().build()
}
pub(super) fn new2(
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>) -> Runtime
{
Runtime {
reactor_handle,
timer_handle,
clock,
executor,
}
}
/// Get a new handle to spawn futures on the single-threaded Tokio runtime
///
/// Different to the runtime itself, the handle can be sent to different
/// threads.
pub fn handle(&self) -> Handle {
Handle(self.executor.handle().clone())
}
/// Spawn a future onto the single-threaded Tokio runtime.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::current_thread::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("running on the runtime");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.spawn(future);
self
}
/// Runs the provided future, blocking the current thread until the future
/// completes.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed. Once the function returns, any uncompleted futures
/// remain pending in the `Runtime` instance. These futures will not run
/// until `block_on` or `run` is called again.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution by calling `block_on` or `run`.
pub fn block_on<F>(&mut self, f: F) -> Result<F::Item, F::Error>
where F: Future
{
self.enter(|executor| {
// Run the provided future
let ret = executor.block_on(f);
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
})
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.enter(|executor| executor.run())
.map_err(|e| RunError {
inner: e,
})
}
fn enter<F, R>(&mut self, f: F) -> R
where F: FnOnce(&mut current_thread::Entered<Timer<Reactor>>) -> R
{
let Runtime {
ref reactor_handle,
ref timer_handle,
ref clock,
ref mut executor,
..
} = *self;
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure
// and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
clock::with_default(clock, enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the
// current single-threaded executor when used. This is a trick,
// because we need two mutable references to the executor (one
// to run the provided future, another to install as the default
// one). We use the fake one here as the default one.
let mut default_executor = current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
f(&mut executor)
})
})
})
})
}
}
+183 -132
View File
@@ -4,6 +4,7 @@
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
@@ -19,11 +20,15 @@
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//! * Run an instance of [`Timer`] **per** thread pool worker thread.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! A timer per thread pool worker thread is used to minimize the amount of
//! synchronization that is required for working with the timer.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
@@ -98,56 +103,61 @@
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../timer/index.html
//! [`Runtime`]: struct.Runtime.html
//! [`Reactor`]: ../reactor/struct.Reactor.html
//! [`ThreadPool`]: ../executor/thread_pool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
//! [`Timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/timer/struct.Timer.html
use reactor::{Reactor, Handle, Background};
mod builder;
pub mod current_thread;
mod shutdown;
mod task_executor;
use tokio_threadpool::{self as threadpool, ThreadPool, Sender};
use futures::Poll;
use futures::future::{self, Future};
pub use self::builder::Builder;
pub use self::shutdown::Shutdown;
pub use self::task_executor::TaskExecutor;
use std::{fmt, io};
use reactor::{Background, Handle};
use std::io;
use tokio_threadpool as threadpool;
use futures;
use futures::future::Future;
#[cfg(feature = "unstable-futures")]
use futures2;
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// Instances of `Runtime` can be created using [`new`] or [`Builder`]. However,
/// most users will use [`tokio::run`], which uses a `Runtime` internally.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
/// [`new`]: #method.new
/// [`Builder`]: struct.Builder.html
/// [`tokio::run`]: fn.run.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
inner: Sender,
}
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
inner: Box<Future<Item = (), Error = ()> + Send>,
}
#[derive(Debug)]
struct Inner {
/// Reactor running on a background thread.
reactor: Background,
/// Task execution pool.
pool: ThreadPool,
pool: threadpool::ThreadPool,
}
// ===== impl Runtime =====
@@ -205,41 +215,97 @@ where F: Future<Item = (), Error = ()> + Send + 'static,
runtime.shutdown_on_idle().wait().unwrap();
}
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// Identical to `run` but works with futures 0.2-style futures.
#[cfg(feature = "unstable-futures")]
pub fn run2<F>(future: F)
where F: futures2::Future<Item = (), Error = futures2::Never> + Send + 'static,
{
let mut runtime = Runtime::new().unwrap();
runtime.spawn2(future);
runtime.shutdown_on_idle().wait().unwrap();
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// This results in a reactor, thread pool, and timer being initialized. The
/// thread pool will not spawn any worker threads until it needs to, i.e.
/// tasks are scheduled to run.
///
/// Most users will not need to call this function directly, instead they
/// will use [`tokio::run`](fn.run.html).
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// Creating a new `Runtime` with default configuration values.
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
// Spawn a reactor on a background thread.
let reactor = Reactor::new()?.background()?;
Builder::new().build()
}
// Get a handle to the reactor.
let handle = reactor.handle().clone();
let pool = threadpool::Builder::new()
.around_worker(move |w, enter| {
::tokio_reactor::with_default(&handle, enter, |_| {
w.run();
});
})
.build();
Ok(Runtime {
inner: Some(Inner {
reactor,
pool,
}),
})
#[deprecated(since = "0.1.5", note = "use `reactor` instead")]
#[doc(hidden)]
pub fn handle(&self) -> &Handle {
self.reactor()
}
/// Return a reference to the reactor handle for this runtime instance.
pub fn handle(&self) -> &Handle {
///
/// The returned handle reference can be cloned in order to get an owned
/// value of the handle. This handle can be used to initialize I/O resources
/// (like TCP or UDP sockets) that will not be used on the runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let reactor_handle = rt.reactor().clone();
///
/// // use `reactor_handle`
/// ```
pub fn reactor(&self) -> &Handle {
self.inner().reactor.handle()
}
/// Return a handle to the runtime's executor.
///
/// The returned handle can be used to spawn tasks that run on this runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let executor_handle = rt.executor();
///
/// // use `executor_handle`
/// ```
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
@@ -287,6 +353,42 @@ impl Runtime {
self
}
/// Spawn a futures 0.2-style future onto the Tokio runtime.
///
/// Otherwise identical to `spawn`
#[cfg(feature = "unstable-futures")]
pub fn spawn2<F>(&mut self, future: F) -> &mut Self
where F: futures2::Future<Item = (), Error = futures2::Never> + Send + 'static,
{
futures2::executor::Executor::spawn(
self.inner_mut().pool.sender_mut(), Box::new(future)
).unwrap();
self
}
/// Run a future to completion on the Tokio runtime.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, and yielding its resolved result. Any tasks or timers which
/// the future spawns internally will be executed on the runtime.
///
/// This method should not be called from an asynchrounous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on<F, R, E>(&mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
rx.wait().unwrap()
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
@@ -302,6 +404,22 @@ impl Runtime {
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_on_idle()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
@@ -336,20 +454,26 @@ impl Runtime {
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_now().and_then(|_| {
reactor.shutdown_now()
})
});
Shutdown { inner }
Shutdown::shutdown_now(inner)
}
fn inner(&self) -> &Inner {
@@ -361,84 +485,11 @@ impl Runtime {
}
}
// ===== impl TaskExecutor =====
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
}
// ===== impl Shutdown =====
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
impl Drop for Runtime {
fn drop(&mut self) {
if let Some(inner) = self.inner.take() {
let shutdown = Shutdown::shutdown_now(inner);
let _ = shutdown.wait();
}
}
}
+46
View File
@@ -0,0 +1,46 @@
use runtime::Inner;
use std::fmt;
use futures::{Future, Poll};
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
pub(super) inner: Box<Future<Item = (), Error = ()> + Send>,
}
impl Shutdown {
pub(super) fn shutdown_now(inner: Inner) -> Self {
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_now().and_then(|_| {
reactor.shutdown_now()
.then(|_| {
Ok(())
})
})
});
Shutdown { inner }
}
}
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
+98
View File
@@ -0,0 +1,98 @@
use tokio_threadpool::Sender;
use futures::future::{self, Future};
#[cfg(feature = "unstable-futures")]
use futures2;
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
pub(super) inner: Sender,
}
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
self.inner.spawn2(future)
}
}
#[cfg(feature = "unstable-futures")]
type Task2 = Box<futures2::Future<Item = (), Error = futures2::Never> + Send>;
#[cfg(feature = "unstable-futures")]
impl futures2::executor::Executor for TaskExecutor {
fn spawn(&mut self, f: Task2) -> Result<(), futures2::executor::SpawnError> {
futures2::executor::Executor::spawn(&mut self.inner, f)
}
fn status(&self) -> Result<(), futures2::executor::SpawnError> {
futures2::executor::Executor::status(&self.inner)
}
}
+86
View File
@@ -0,0 +1,86 @@
//! Utilities for tracking time.
//!
//! This module provides a number of types for executing code after a set period
//! of time.
//!
//! * [`Delay`][Delay] is a future that does no work and completes at a specific `Instant`
//! in time.
//!
//! * [`Interval`][Interval] is a stream yielding a value at a fixed period. It
//! is initialized with a `Duration` and repeatedly yields each time the
//! duration elapses.
//!
//! * [`Deadline`][Deadline] wraps a future, requiring that it completes before
//! a specified `Instant` in time. If the future does not complete in time,
//! then it is canceled and an error is returned.
//!
//! These types are sufficient for handling a large number of scenarios
//! involving time.
//!
//! These types must be used from within the context of the
//! [`Runtime`][runtime] or a timer context must be setup explicitly. See the
//! [`tokio-timer`][tokio-timer] crate for more details on how to setup a timer
//! context.
//!
//! # Examples
//!
//! Wait 100ms and print "Hello World!"
//!
//! ```
//! use tokio::prelude::*;
//! use tokio::timer::Delay;
//!
//! use std::time::{Duration, Instant};
//!
//! let when = Instant::now() + Duration::from_millis(100);
//!
//! tokio::run({
//! Delay::new(when)
//! .map_err(|e| panic!("timer failed; err={:?}", e))
//! .and_then(|_| {
//! println!("Hello world!");
//! Ok(())
//! })
//! })
//! ```
//!
//! Require that an operation takes no more than 300ms. Note that this uses the
//! [`deadline`][ext] function on the [`FutureExt`][ext] trait. This trait is
//! included in the prelude.
//!
//! ```
//! # extern crate futures;
//! # extern crate tokio;
//! use tokio::prelude::*;
//!
//! use std::time::{Duration, Instant};
//!
//! fn long_op() -> Box<Future<Item = (), Error = ()> + Send> {
//! // ...
//! # Box::new(futures::future::ok(()))
//! }
//!
//! # fn main() {
//! let when = Instant::now() + Duration::from_millis(300);
//!
//! tokio::run({
//! long_op()
//! .deadline(when)
//! .map_err(|e| {
//! println!("operation timed out");
//! })
//! })
//! # }
//! ```
//!
//! [runtime]: ../runtime/struct.Runtime.html
//! [tokio-timer]: https://docs.rs/tokio-timer
//! [ext]: ../util/trait.FutureExt.html#method.deadline
pub use tokio_timer::{
Deadline,
DeadlineError,
Error,
Interval,
Delay,
};
+61
View File
@@ -0,0 +1,61 @@
use tokio_timer::Deadline;
use futures::Future;
use std::time::Instant;
/// An extension trait for `Future` that provides a variety of convenient
/// combinator functions.
///
/// Currently, there only is a [`deadline`] function, but this will increase
/// over time.
///
/// Users are not expected to implement this trait. All types that implement
/// `Future` already implement `FutureExt`.
///
/// This trait can be imported directly or via the Tokio prelude: `use
/// tokio::prelude::*`.
///
/// [`deadline`]: #method.deadline
pub trait FutureExt: Future {
/// Creates a new future which allows `self` until `deadline`.
///
/// This combinator creates a new future which wraps the receiving future
/// with a deadline. The returned future is allowed to execute until it
/// completes or `deadline` is reached, whichever happens first.
///
/// If the future completes before `deadline` then the future will resolve
/// with that item. Otherwise the future will resolve to an error once
/// `deadline` is reached.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// use tokio::prelude::*;
/// use std::time::{Duration, Instant};
/// # use futures::future::{self, FutureResult};
///
/// # fn long_future() -> FutureResult<(), ()> {
/// # future::ok(())
/// # }
/// #
/// # fn main() {
/// let future = long_future()
/// .deadline(Instant::now() + Duration::from_secs(1))
/// .map_err(|e| println!("error = {:?}", e));
///
/// tokio::run(future);
/// # }
/// ```
fn deadline(self, deadline: Instant) -> Deadline<Self>
where Self: Sized,
{
Deadline::new(self, deadline)
}
}
impl<T: ?Sized> FutureExt for T where T: Future {}
+9
View File
@@ -0,0 +1,9 @@
//! Utilities for working with Tokio.
//!
//! This module contains utilities that are useful for working with Tokio.
//! Currently, this only includes [`FutureExt`][FutureExt]. However, this will
//! include over time.
mod future;
pub use self::future::FutureExt;
+69
View File
@@ -0,0 +1,69 @@
extern crate futures;
extern crate tokio;
extern crate tokio_timer;
extern crate env_logger;
use tokio::prelude::*;
use tokio::runtime::{self, current_thread};
use tokio::timer::*;
use tokio_timer::clock::Clock;
use std::sync::mpsc;
use std::time::{Duration, Instant};
struct MockNow(Instant);
impl tokio_timer::clock::Now for MockNow {
fn now(&self) -> Instant {
self.0
}
}
#[test]
fn clock_and_timer_concurrent() {
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = runtime::Builder::new()
.clock(clock)
.build()
.unwrap();
let (tx, rx) = mpsc::channel();
rt.spawn({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn clock_and_timer_single_threaded() {
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = current_thread::Builder::new()
.clock(clock)
.build()
.unwrap();
rt.block_on({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
Ok(())
})
}).unwrap();
}
-395
View File
@@ -1,395 +0,0 @@
extern crate tokio;
extern crate tokio_executor;
extern crate futures;
use tokio::executor::current_thread::{self, block_on_all, CurrentThread};
use std::any::Any;
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::prelude::*;
use futures::sync::oneshot;
#[test]
fn spawn_from_block_on_all() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let msg = current_thread::block_on_all(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Rc::new(Cell::new(0));
let cnt2 = cnt.clone();
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
assert_eq!(1, cnt2.get());
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
for _ in 0..ITER {
let cnt = cnt.clone();
current_thread.spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok::<(), ()>(())
}));
}
current_thread.run().unwrap();
assert_eq!(cnt.get(), ITER);
}
#[test]
fn does_not_set_global_executor_by_default() {
use tokio_executor::Executor;
block_on_all(lazy(|| {
tokio_executor::DefaultExecutor::current()
.spawn(Box::new(lazy(|| ok())))
.unwrap_err();
ok()
})).unwrap();
}
#[test]
fn spawn_from_block_on_future() {
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
current_thread::spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
}));
Ok::<_, ()>(())
})).unwrap();
current_thread.run().unwrap();
assert_eq!(1, cnt.get());
}
struct Never(Rc<()>);
impl Future for Never {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(Async::NotReady)
}
}
#[test]
fn outstanding_tasks_are_dropped_when_executor_is_dropped() {
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.spawn(Never(rc.clone()));
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
// Using the global spawn fn
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
current_thread::spawn(Never(rc.clone()));
Ok::<_, ()>(())
})).unwrap();
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
}
#[test]
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
})).unwrap();
}
#[test]
#[should_panic]
fn run_in_future() {
block_on_all(lazy(|| {
current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
}));
ok()
})).unwrap();
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
struct Infini {
num: Rc<Cell<usize>>,
}
impl Future for Infini {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.num.set(1 + self.num.get());
task::current().notify();
Ok(Async::NotReady)
}
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.turn(None)
.unwrap();
assert_eq!(1, num.get());
}
#[test]
fn tasks_are_scheduled_fairly() {
let state = Rc::new(RefCell::new([0, 0]));
struct Spin {
state: Rc<RefCell<[i32; 2]>>,
idx: usize,
}
impl Future for Spin {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
let mut state = self.state.borrow_mut();
if self.idx == 0 {
let diff = state[0] - state[1];
assert!(diff.abs() <= 1);
if state[0] >= 50 {
return Ok(().into());
}
}
state[self.idx] += 1;
if state[self.idx] >= 100 {
return Ok(().into());
}
task::current().notify();
Ok(Async::NotReady)
}
}
block_on_all(lazy(|| {
current_thread::spawn(Spin {
state: state.clone(),
idx: 0,
});
current_thread::spawn(Spin {
state: state,
idx: 1,
});
ok()
})).unwrap();
}
#[test]
fn spawn_and_turn() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let mut current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
current_thread.spawn(lazy(move || {
Ok(())
}));
// Turn once...
current_thread.turn(None).unwrap();
current_thread.spawn(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok(())
}));
// This does not run the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(1, cnt.get());
// This runs the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(2, cnt.get());
}
#[test]
fn spawn_in_drop() {
let mut current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
current_thread.spawn({
struct OnDrop<F: FnOnce()>(Option<F>);
impl<F: FnOnce()> Drop for OnDrop<F> {
fn drop(&mut self) {
(self.0.take().unwrap())();
}
}
struct MyFuture {
_data: Box<Any>,
}
impl Future for MyFuture {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(().into())
}
}
MyFuture {
_data: Box::new(OnDrop(Some(move || {
current_thread::spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}));
}))),
}
});
current_thread.block_on(rx).unwrap();
current_thread.run().unwrap();
}
#[test]
fn hammer_turn() {
use futures::sync::mpsc;
const ITER: usize = 100;
const N: usize = 100;
const THREADS: usize = 4;
for _ in 0..ITER {
let mut ths = vec![];
// Add some jitter
for _ in 0..THREADS {
let th = thread::spawn(|| {
let mut current_thread = CurrentThread::new();
let (tx, rx) = mpsc::unbounded();
current_thread.spawn({
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
rx.for_each(move |_| {
c.set(1 + c.get());
Ok(())
})
.map_err(|e| panic!("err={:?}", e))
.map(move |v| {
assert_eq!(N, cnt.get());
v
})
});
thread::spawn(move || {
for _ in 0..N {
tx.unbounded_send(()).unwrap();
thread::yield_now();
}
});
while !current_thread.is_idle() {
current_thread.turn(None).unwrap();
}
});
ths.push(th);
}
for th in ths {
th.join().unwrap();
}
}
}
fn ok() -> future::FutureResult<(), ()> {
future::ok(())
}
+53
View File
@@ -0,0 +1,53 @@
#![cfg(feature = "unstable-futures")]
// This test is the same as `echo.rs`, but ported to futures 0.2
extern crate env_logger;
extern crate futures2;
extern crate tokio;
extern crate tokio_io;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::thread;
use futures2::prelude::*;
use futures2::executor::block_on;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = TcpStream::connect(&addr).unwrap();
for _i in 0..1024 {
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
let mut buf = [0; 1024];
assert_eq!(t!(s.read(&mut buf)), msg.len());
assert_eq!(&buf[..msg.len()], msg.as_bytes());
}
});
let clients = srv.incoming();
let client = clients.next().map(|e| e.0.unwrap()).map_err(|e| e.0);
let halves = client.map(|s| s.split());
let copied = halves.and_then(|(a, b)| a.copy_into(b));
let (amt, _, _) = t!(block_on(copied));
t.join().unwrap();
assert_eq!(amt, msg.len() as u64 * 1024);
}
+48 -32
View File
@@ -5,6 +5,8 @@ extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
@@ -18,7 +20,7 @@ macro_rules! t {
}
#[test]
fn hammer() {
fn hammer_old() {
let _ = env_logger::init();
let threads = (0..10).map(|_| {
@@ -73,48 +75,62 @@ fn hammer_split() {
use tokio_io::io;
const N: usize = 100;
const ITER: usize = 10;
let _ = env_logger::init();
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
for _ in 0..ITER {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mut rt = Runtime::new().unwrap();
let cnt = Arc::new(AtomicUsize::new(0));
fn split(socket: TcpStream) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let mut rt = Runtime::new().unwrap();
let rd = io::read(rd, vec![0; 1])
.map(|_| ())
.map_err(|e| panic!("read error = {:?}", e));
fn split(socket: TcpStream, cnt: Arc<AtomicUsize>) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let wr = io::write_all(wr, b"1")
.map(|_| ())
.map_err(|e| panic!("write error = {:?}", e));
let cnt2 = cnt.clone();
tokio::spawn(rd);
tokio::spawn(wr);
}
let rd = io::read(rd, vec![0; 1])
.map(move |_| {
cnt2.fetch_add(1, Relaxed);
})
.map_err(|e| panic!("read error = {:?}", e));
rt.spawn({
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(|socket| {
split(socket);
Ok(())
})
});
let wr = io::write_all(wr, b"1")
.map(move |_| {
cnt.fetch_add(1, Relaxed);
})
.map_err(move |e| panic!("write error = {:?}", e));
tokio::spawn(rd);
tokio::spawn(wr);
}
for _ in 0..N {
rt.spawn({
TcpStream::connect(&addr)
.map_err(|e| panic!("connect error = {:?}", e))
.map(|socket| split(socket))
let cnt = cnt.clone();
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(move |socket| {
split(socket, cnt.clone());
Ok(())
})
});
}
rt.shutdown_on_idle().wait().unwrap();
for _ in 0..N {
rt.spawn({
let cnt = cnt.clone();
TcpStream::connect(&addr)
.map_err(move |e| panic!("connect error = {:?}", e))
.map(move |socket| split(socket, cnt))
});
}
rt.shutdown_on_idle().wait().unwrap();
assert_eq!(N * 4, cnt.load(Relaxed));
}
}
+122
View File
@@ -0,0 +1,122 @@
#![cfg(feature = "unstable-futures")]
// This test is the same as `global.rs`, but ported to futures 0.2
extern crate futures;
extern crate futures2;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use futures2::prelude::*;
use futures2::executor::block_on;
use futures2::task;
use tokio::net::{TcpStream, TcpListener};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn hammer() {
let _ = env_logger::init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().next()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(block_on(mine.join(theirs)));
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
}).collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
}
struct Rd(Arc<TcpStream>);
struct Wr(Arc<TcpStream>);
impl AsyncRead for Rd {
fn poll_read(&mut self, cx: &mut task::Context, dst: &mut [u8]) -> Poll<usize, io::Error> {
<&TcpStream>::poll_read(&mut &*self.0, cx, dst)
}
}
impl AsyncWrite for Wr {
fn poll_write(&mut self, cx: &mut task::Context, src: &[u8]) -> Poll<usize, io::Error> {
<&TcpStream>::poll_write(&mut &*self.0, cx, src)
}
fn poll_flush(&mut self, _cx: &mut task::Context) -> Poll<(), io::Error> {
Ok(().into())
}
fn poll_close(&mut self, _cx: &mut task::Context) -> Poll<(), io::Error> {
Ok(().into())
}
}
#[test]
fn hammer_split() {
const N: usize = 100;
let _ = env_logger::init();
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mut rt = Runtime::new().unwrap();
fn split(socket: TcpStream) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let rd = rd.read(vec![0; 1])
.map(|_| ())
.map_err(|e| panic!("read error = {:?}", e));
let wr = wr.write_all(b"1")
.map(|_| ())
.map_err(|e| panic!("write error = {:?}", e));
tokio::spawn2(rd);
tokio::spawn2(wr);
}
rt.spawn2({
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(|socket| {
split(socket);
Ok(())
})
.map(|_| ())
});
for _ in 0..N {
rt.spawn2({
TcpStream::connect(&addr)
.map_err(|e| panic!("connect error = {:?}", e))
.map(|socket| split(socket))
});
}
futures::Future::wait(rt.shutdown_on_idle()).unwrap();
}
+11 -10
View File
@@ -1,8 +1,9 @@
extern crate env_logger;
extern crate futures;
extern crate futures_cpupool;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate tokio_threadpool;
extern crate bytes;
use std::io;
@@ -10,12 +11,10 @@ use std::net::Shutdown;
use bytes::{BytesMut, BufMut};
use futures::{Future, Stream, Sink};
use futures::future::Executor;
use futures_cpupool::CpuPool;
use tokio::net::{TcpListener, TcpStream};
use tokio_io::codec::{Encoder, Decoder};
use tokio_codec::{Encoder, Decoder};
use tokio_io::io::{write_all, read};
use tokio_io::AsyncRead;
use tokio_threadpool::Builder;
pub struct LineCodec;
@@ -54,18 +53,20 @@ impl Encoder for LineCodec {
fn echo() {
drop(env_logger::init());
let pool = CpuPool::new(1);
let pool = Builder::new()
.pool_size(1)
.build();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
let addr = listener.local_addr().unwrap();
let pool_inner = pool.clone();
let sender = pool.sender().clone();
let srv = listener.incoming().for_each(move |socket| {
let (sink, stream) = socket.framed(LineCodec).split();
pool_inner.execute(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
let (sink, stream) = LineCodec.framed(socket).split();
sender.spawn(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
Ok(())
});
pool.execute(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
pool.sender().spawn(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
let client = TcpStream::connect(&addr);
let client = client.wait().unwrap();
+89
View File
@@ -0,0 +1,89 @@
extern crate futures;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_tcp;
use tokio_reactor::Reactor;
use tokio_tcp::TcpListener;
use futures::{Future, Stream};
use futures::executor::{spawn, Notify, Spawn};
use std::mem;
use std::net::TcpStream;
use std::sync::{Arc, Mutex};
#[test]
fn test_drop_on_notify() {
// When the reactor receives a kernel notification, it notifies the
// task that holds the associated socket. If this notification results in
// the task being dropped, the socket will also be dropped.
//
// Previously, there was a deadlock scenario where the reactor, while
// notifying, held a lock and the task being dropped attempted to acquire
// that same lock in order to clean up state.
//
// To simulate this case, we create a fake executor that does nothing when
// the task is notified. This simulates an executor in the process of
// shutting down. Then, when the task handle is dropped, the task itself is
// dropped.
struct MyNotify;
type Task = Mutex<Spawn<Box<Future<Item = (), Error = ()>>>>;
impl Notify for MyNotify {
fn notify(&self, _: usize) {
// Do nothing
}
fn clone_id(&self, id: usize) -> usize {
let ptr = id as *const Task;
let task = unsafe { Arc::from_raw(ptr) };
mem::forget(task.clone());
mem::forget(task);
id
}
fn drop_id(&self, id: usize) {
let ptr = id as *const Task;
let _ = unsafe { Arc::from_raw(ptr) };
}
}
let addr = "127.0.0.1:0".parse().unwrap();
let mut reactor = Reactor::new().unwrap();
// Create a listener
let listener = TcpListener::bind(&addr).unwrap();
let addr = listener.local_addr().unwrap();
// Define a task that just drains the listener
let task = Box::new({
listener.incoming()
.for_each(|_| Ok(()))
.map_err(|_| panic!())
}) as Box<Future<Item = (), Error = ()>>;
let task = Arc::new(Mutex::new(spawn(task)));
let notify = Arc::new(MyNotify);
let mut enter = tokio_executor::enter().unwrap();
tokio_reactor::with_default(&reactor.handle(), &mut enter, |_| {
let id = &*task as *const Task as usize;
task.lock().unwrap()
.poll_future_notify(&notify, id)
.unwrap();
});
drop(task);
// Establish a connection to the acceptor
let _s = TcpStream::connect(&addr).unwrap();
reactor.turn(None).unwrap();
}
+159 -36
View File
@@ -1,11 +1,15 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
extern crate futures;
use futures::prelude::*;
use futures::sync::oneshot;
use std::sync::{Arc, Mutex};
use std::thread;
use tokio::io;
use tokio::net::{TcpStream, TcpListener};
use tokio_io::io;
use tokio::prelude::future::lazy;
use tokio::prelude::*;
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
@@ -14,39 +18,158 @@ macro_rules! t {
})
}
fn create_client_server_future() -> Box<Future<Item=(), Error=()> + Send> {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| panic!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| ())
.map_err(|e| panic!("write err = {:?}", e))
})
})
.map(|_| ());
let client = client
.map_err(|e| panic!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| ())
.map_err(|e| panic!("read err = {:?}", e))
});
let future = server.join(client)
.map(|_| ());
Box::new(future)
}
#[test]
fn basic_runtime_usage() {
fn runtime_tokio_run() {
let _ = env_logger::init();
// TODO: Don't require the lazy wrapper
tokio::run(::futures::future::lazy(|| {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| println!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| println!("write done"))
.map_err(|e| println!("write err = {:?}", e))
})
})
.map(|_| println!("accept done"));
let client = client
.map_err(|e| println!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| println!("read done"))
.map_err(|e| println!("read err = {:?}", e))
});
tokio::spawn({
server.join(client)
.map(|_| println!("done"))
})
}));
tokio::run(create_client_server_future());
}
#[test]
fn runtime_single_threaded() {
let _ = env_logger::init();
let mut runtime = tokio::runtime::current_thread::Runtime::new()
.unwrap();
runtime.block_on(create_client_server_future()).unwrap();
runtime.run().unwrap();
}
#[test]
fn runtime_multi_threaded() {
let _ = env_logger::init();
let mut runtime = tokio::runtime::Builder::new()
.build()
.unwrap();
runtime.spawn(create_client_server_future());
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn block_on_timer() {
use std::time::{Duration, Instant};
use tokio::timer::{Delay, Error};
fn after_1s<T>(x: T) -> Box<Future<Item = T, Error = Error> + Send>
where
T: Send + 'static,
{
Box::new(Delay::new(Instant::now() + Duration::from_millis(100)).map(move |_| x))
}
let mut runtime = Runtime::new().unwrap();
assert_eq!(runtime.block_on(after_1s(42)).unwrap(), 42);
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn_from_block_on() {
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
let msg = runtime
.block_on(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
tokio::spawn(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
}))
.unwrap();
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
use std::time::Duration;
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
runtime
.block_on(rx.then(move |_| {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<_, ()>(())
}))
.unwrap();
assert_eq!(1, *cnt.lock().unwrap());
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Arc::new(Mutex::new(0));
let mut runtime = Runtime::new().unwrap();
for _ in 0..ITER {
let c = cnt.clone();
runtime.spawn(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
}));
}
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(ITER, *cnt.lock().unwrap());
}
+136
View File
@@ -0,0 +1,136 @@
#![cfg(feature = "unstable-futures")]
// This test is the same as `tcp.rs`, but ported to futures 0.2
extern crate env_logger;
extern crate tokio;
extern crate mio;
extern crate futures2;
use std::{net, thread};
use std::sync::mpsc::channel;
use tokio::net::{TcpListener, TcpStream};
use futures2::executor::block_on;
use futures2::prelude::*;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn connect() {
drop(env_logger::init());
let srv = t!(net::TcpListener::bind("127.0.0.1:0"));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
t!(srv.accept()).0
});
let stream = TcpStream::connect(&addr);
let mine = t!(block_on(stream));
let theirs = t.join().unwrap();
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
}
#[test]
fn accept() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let (tx, rx) = channel();
let client = srv.incoming().map(move |t| {
tx.send(()).unwrap();
t
}).next().map_err(|e| e.0);
assert!(rx.try_recv().is_err());
let t = thread::spawn(move || {
net::TcpStream::connect(&addr).unwrap()
});
let (mine, _remaining) = t!(block_on(client));
let mine = mine.unwrap();
let theirs = t.join().unwrap();
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
}
#[test]
fn accept2() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
net::TcpStream::connect(&addr).unwrap()
});
let (tx, rx) = channel();
let client = srv.incoming().map(move |t| {
tx.send(()).unwrap();
t
}).next().map_err(|e| e.0);
assert!(rx.try_recv().is_err());
let (mine, _remaining) = t!(block_on(client));
mine.unwrap();
t.join().unwrap();
}
#[cfg(unix)]
mod unix {
use tokio::net::TcpStream;
use tokio::prelude::*;
use env_logger;
use futures2::future;
use futures2::executor::block_on;
use futures2::io::AsyncRead;
use mio::unix::UnixReady;
use std::{net, thread};
use std::time::Duration;
#[test]
fn poll_hup() {
drop(env_logger::init());
let srv = t!(net::TcpListener::bind("127.0.0.1:0"));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
let mut client = t!(srv.accept()).0;
client.write(b"hello world").unwrap();
thread::sleep(Duration::from_millis(200));
});
let mut stream = t!(block_on(TcpStream::connect(&addr)));
// Poll for HUP before reading.
block_on(future::poll_fn(|cx| {
stream.poll_read_ready2(cx, UnixReady::hup().into())
})).unwrap();
// Same for write half
block_on(future::poll_fn(|cx| {
stream.poll_write_ready2(cx)
})).unwrap();
let mut buf = vec![0; 11];
// Read the data
block_on(future::poll_fn(|cx| {
stream.poll_read(cx, &mut buf)
})).unwrap();
assert_eq!(b"hello world", &buf[..]);
t.join().unwrap();
}
}
+94
View File
@@ -0,0 +1,94 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use tokio::prelude::*;
use tokio::timer::*;
use std::sync::mpsc;
use std::time::{Duration, Instant};
#[test]
fn timer_with_runtime() {
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(100);
let (tx, rx) = mpsc::channel();
tokio::run({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn starving() {
use futures::{task, Poll, Async};
let _ = env_logger::init();
struct Starve(Delay, u64);
impl Future for Starve {
type Item = u64;
type Error = ();
fn poll(&mut self) -> Poll<Self::Item, ()> {
if self.0.poll().unwrap().is_ready() {
return Ok(self.1.into());
}
self.1 += 1;
task::current().notify();
Ok(Async::NotReady)
}
}
let when = Instant::now() + Duration::from_millis(20);
let starve = Starve(Delay::new(when), 0);
let (tx, rx) = mpsc::channel();
tokio::run({
starve
.and_then(move |_ticks| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn deadline() {
use futures::future;
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(20);
let (tx, rx) = mpsc::channel();
tokio::run({
future::empty::<(), ()>()
.deadline(when)
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
+3
View File
@@ -0,0 +1,3 @@
# # 0.1.0 (June 13, 2018)
* Initial release (#353)
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "tokio-codec"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>", "Bryan Burgers <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-codec/0.1"
description = """
Utilities for encoding and decoding frames.
"""
categories = ["asynchronous"]
[dependencies]
tokio-io = { version = "0.1.7", path = "../tokio-io" }
bytes = "0.4.7"
futures = "0.1.18"
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2018 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+35
View File
@@ -0,0 +1,35 @@
# tokio-codec
Utilities for encoding and decoding frames.
[Documentation](https://docs.rs/tokio-codec)
## Usage
First, add this to your `Cargo.toml`:
```toml
[dependencies]
tokio-codec = "0.1"
```
Next, add this to your crate:
```rust
extern crate tokio_codec;
```
You can find extensive documentation and examples about how to use this crate
online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-codec) is also a great place to get started
for the nitty-gritty.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+37
View File
@@ -0,0 +1,37 @@
use bytes::{Bytes, BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::io;
/// A simple `Codec` implementation that just ships bytes around.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
pub struct BytesCodec(());
impl BytesCodec {
/// Creates a new `BytesCodec` for shipping around raw bytes.
pub fn new() -> BytesCodec { BytesCodec(()) }
}
impl Decoder for BytesCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<BytesMut>, io::Error> {
if buf.len() > 0 {
let len = buf.len();
Ok(Some(buf.split_to(len)))
} else {
Ok(None)
}
}
}
impl Encoder for BytesCodec {
type Item = Bytes;
type Error = io::Error;
fn encode(&mut self, data: Bytes, buf: &mut BytesMut) -> Result<(), io::Error> {
buf.reserve(data.len());
buf.put(data);
Ok(())
}
}
+32
View File
@@ -0,0 +1,32 @@
//! Utilities for encoding and decoding frames.
//!
//! Contains adapters to go from streams of bytes, [`AsyncRead`] and
//! [`AsyncWrite`], to framed streams implementing [`Sink`] and [`Stream`].
//! Framed streams are also known as [transports].
//!
//! [`AsyncRead`]: #
//! [`AsyncWrite`]: #
//! [`Sink`]: #
//! [`Stream`]: #
//! [transports]: #
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-codec/0.1.0")]
extern crate bytes;
extern crate tokio_io;
mod bytes_codec;
mod lines_codec;
pub use tokio_io::_tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
};
pub use bytes_codec::BytesCodec;
pub use lines_codec::LinesCodec;
+89
View File
@@ -0,0 +1,89 @@
use bytes::{BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::{io, str};
/// A simple `Codec` implementation that splits up data into lines.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
pub struct LinesCodec {
// Stored index of the next index to examine for a `\n` character.
// This is used to optimize searching.
// For example, if `decode` was called with `abc`, it would hold `3`,
// because that is the next index to examine.
// The next time `decode` is called with `abcde\n`, the method will
// only look at `de\n` before returning.
next_index: usize,
}
impl LinesCodec {
/// Returns a `LinesCodec` for splitting up data into lines.
pub fn new() -> LinesCodec {
LinesCodec { next_index: 0 }
}
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf).map_err(|_|
io::Error::new(
io::ErrorKind::InvalidData,
"Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
if let Some(&b'\r') = s.last() {
&s[..s.len() - 1]
} else {
s
}
}
impl Decoder for LinesCodec {
type Item = String;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<String>, io::Error> {
if let Some(newline_offset) =
buf[self.next_index..].iter().position(|b| *b == b'\n')
{
let newline_index = newline_offset + self.next_index;
let line = buf.split_to(newline_index + 1);
let line = &line[..line.len()-1];
let line = without_carriage_return(line);
let line = utf8(line)?;
self.next_index = 0;
Ok(Some(line.to_string()))
} else {
self.next_index = buf.len();
Ok(None)
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> Result<Option<String>, io::Error> {
Ok(match self.decode(buf)? {
Some(frame) => Some(frame),
None => {
// No terminating newline - return remaining data, if any
if buf.is_empty() || buf == &b"\r"[..] {
None
} else {
let line = buf.take();
let line = without_carriage_return(&line);
let line = utf8(line)?;
self.next_index = 0;
Some(line.to_string())
}
}
})
}
}
impl Encoder for LinesCodec {
type Item = String;
type Error = io::Error;
fn encode(&mut self, line: String, buf: &mut BytesMut) -> Result<(), io::Error> {
buf.reserve(line.len() + 1);
buf.put(line);
buf.put_u8(b'\n');
Ok(())
}
}
@@ -1,8 +1,8 @@
extern crate tokio_io;
extern crate tokio_codec;
extern crate bytes;
use bytes::{BytesMut, Bytes, BufMut};
use tokio_io::codec::{BytesCodec, LinesCodec, Decoder, Encoder};
use tokio_codec::{BytesCodec, LinesCodec, Decoder, Encoder};
#[test]
fn bytes_decoder() {
@@ -1,15 +1,17 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
use futures::{Stream, Future};
use std::io::{self, Read};
use tokio_io::codec::{Framed, FramedParts, Decoder, Encoder};
use tokio_codec::{Framed, FramedParts, Decoder, Encoder};
use tokio_io::AsyncRead;
use bytes::{BytesMut, Buf, BufMut, IntoBuf, BigEndian};
use bytes::{BytesMut, Buf, BufMut, IntoBuf};
const INITIAL_CAPACITY: usize = 8 * 1024;
/// Encode and decode u32 values.
struct U32Codec;
impl Decoder for U32Codec {
@@ -21,7 +23,7 @@ impl Decoder for U32Codec {
return Ok(None);
}
let n = buf.split_to(4).into_buf().get_u32::<BigEndian>();
let n = buf.split_to(4).into_buf().get_u32_be();
Ok(Some(n))
}
}
@@ -33,11 +35,12 @@ impl Encoder for U32Codec {
fn encode(&mut self, item: u32, dst: &mut BytesMut) -> io::Result<()> {
// Reserve space
dst.reserve(4);
dst.put_u32::<BigEndian>(item);
dst.put_u32_be(item);
Ok(())
}
}
/// This value should never be used
struct DontReadIntoThis;
impl Read for DontReadIntoThis {
@@ -51,12 +54,10 @@ impl AsyncRead for DontReadIntoThis {}
#[test]
fn can_read_from_existing_buf() {
let parts = FramedParts {
inner: DontReadIntoThis,
readbuf: vec![0, 0, 0, 42].into(),
writebuf: BytesMut::with_capacity(0),
};
let framed = Framed::from_parts(parts, U32Codec);
let mut parts = FramedParts::new(DontReadIntoThis, U32Codec);
parts.read_buf = vec![0, 0, 0, 42].into();
let framed = Framed::from_parts(parts);
let num = framed
.into_future()
@@ -66,32 +67,28 @@ fn can_read_from_existing_buf() {
.wait()
.map_err(|e| e.0)
.unwrap();
assert_eq!(num, 42);
}
#[test]
fn external_buf_grows_to_init() {
let parts = FramedParts {
inner: DontReadIntoThis,
readbuf: vec![0, 0, 0, 42].into(),
writebuf: BytesMut::with_capacity(0),
};
let framed = Framed::from_parts(parts, U32Codec);
let FramedParts { readbuf, .. } = framed.into_parts();
let mut parts = FramedParts::new(DontReadIntoThis, U32Codec);
parts.read_buf = vec![0, 0, 0, 42].into();
assert_eq!(readbuf.capacity(), INITIAL_CAPACITY);
let framed = Framed::from_parts(parts);
let FramedParts { read_buf, .. } = framed.into_parts();
assert_eq!(read_buf.capacity(), INITIAL_CAPACITY);
}
#[test]
fn external_buf_does_not_shrink() {
let parts = FramedParts {
inner: DontReadIntoThis,
readbuf: vec![0; INITIAL_CAPACITY * 2].into(),
writebuf: BytesMut::with_capacity(0),
};
let framed = Framed::from_parts(parts, U32Codec);
let FramedParts { readbuf, .. } = framed.into_parts();
let mut parts = FramedParts::new(DontReadIntoThis, U32Codec);
parts.read_buf = vec![0; INITIAL_CAPACITY * 2].into();
assert_eq!(readbuf.capacity(), INITIAL_CAPACITY * 2);
let framed = Framed::from_parts(parts);
let FramedParts { read_buf, .. } = framed.into_parts();
assert_eq!(read_buf.capacity(), INITIAL_CAPACITY * 2);
}
@@ -1,9 +1,10 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
use tokio_io::AsyncRead;
use tokio_io::codec::{FramedRead, Decoder};
use tokio_codec::{FramedRead, Decoder};
use bytes::{BytesMut, Buf, IntoBuf, BigEndian};
use futures::Stream;
@@ -1,9 +1,10 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
use tokio_io::AsyncWrite;
use tokio_io::codec::{Encoder, FramedWrite};
use tokio_codec::{Encoder, FramedWrite};
use futures::{Sink, Poll};
use bytes::{BytesMut, BufMut, BigEndian};
@@ -28,7 +29,7 @@ impl Encoder for U32Encoder {
fn encode(&mut self, item: u32, dst: &mut BytesMut) -> io::Result<()> {
// Reserve space
dst.reserve(4);
dst.put_u32::<BigEndian>(item);
dst.put_u32_be(item);
Ok(())
}
}
@@ -65,7 +66,7 @@ fn write_hits_backpressure() {
for i in 0..(ITER + 1) {
let mut b = BytesMut::with_capacity(4);
b.put_u32::<BigEndian>(i as u32);
b.put_u32_be(i as u32);
// Append to the end
match mock.calls.back_mut().unwrap() {
+3
View File
@@ -0,0 +1,3 @@
# 0.1.0 (June 13, 2018)
* Extract `tokio::executor::current_thread` to a tokio-current-thread crate (#356)
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "tokio-current-thread"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
documentation = "https://docs.rs/tokio-current-thread"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
authors = ["Carl Lerche <[email protected]>"]
description = """
Single threaded executor which manage many tasks concurrently on the current thread.
"""
keywords = ["futures", "tokio"]
categories = ["concurrency", "asynchronous"]
[dependencies]
tokio-executor = { version = "0.1.2", path = "../tokio-executor" }
futures = "0.1.19"
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2018 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+19
View File
@@ -0,0 +1,19 @@
# tokio-current-thread
Single threaded executor for Tokio.
[Documentation](https://tokio-rs.github.io/tokio/tokio_current_thread/)
## Overview
This crate provides the single threaded executor which execute many tasks concurrently.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+709
View File
@@ -0,0 +1,709 @@
//! A single-threaded executor which executes tasks on the same thread from which
//! they are spawned.
//!
//!
//! The crate provides:
//!
//! * [`CurrentThread`] is the main type of this crate. It executes tasks on the current thread.
//! The easiest way to start a new [`CurrentThread`] executor is to call
//! [`block_on_all`] with an initial task to seed the executor.
//! All tasks that are being managed by a [`CurrentThread`] executor are able to
//! spawn additional tasks by calling [`spawn`].
//!
//!
//! Application authors will not use this crate directly. Instead, they will use the
//! `tokio` crate. Library authors should only depend on `tokio-current-thread` if they
//! are building a custom task executor.
//!
//! For more details, see [executor module] documentation in the Tokio crate.
//!
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`spawn`]: fn.spawn.html
//! [`block_on_all`]: fn.block_on_all.html
//! [executor module]: https://docs.rs/tokio/0.1/tokio/executor/index.html
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.0")]
#![deny(warnings, missing_docs, missing_debug_implementations)]
extern crate futures;
extern crate tokio_executor;
mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::{Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::{executor, Async, Future};
use futures::future::{Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::rc::Rc;
use std::time::{Duration, Instant};
use std::sync::mpsc;
#[cfg(feature = "unstable-futures")]
use futures2;
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
/// Execute futures and receive unpark notifications.
scheduler: Scheduler<P::Unpark>,
/// Current number of futures being executed
num_futures: usize,
/// Thread park handle
park: P,
/// Handle for spawning new futures from other threads
spawn_handle: Handle,
/// Receiver for futures spawned from other threads
spawn_receiver: mpsc::Receiver<Box<Future<Item = (), Error = ()> + Send + 'static>>,
}
/// Executes futures on the current thread.
///
/// All futures executed using this executor will be executed on the current
/// thread. As such, `run` will wait for these futures to complete before
/// returning.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
// Prevent the handle from moving across threads.
_p: ::std::marker::PhantomData<Rc<()>>,
}
/// Returned by the `turn` function.
#[derive(Debug)]
pub struct Turn {
polled: bool
}
impl Turn {
/// `true` if any futures were polled at all and `false` otherwise.
pub fn has_polled(&self) -> bool {
self.polled
}
}
/// A `CurrentThread` instance bound to a supplied execution context.
pub struct Entered<'a, P: Park + 'a> {
executor: &'a mut CurrentThread<P>,
enter: &'a mut Enter,
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
_p: (),
}
/// Error returned by the `run_timeout` function.
#[derive(Debug)]
pub struct RunTimeoutError {
timeout: bool,
}
/// Error returned by the `turn` function.
#[derive(Debug)]
pub struct TurnError {
_p: (),
}
/// Error returned by the `block_on` function.
#[derive(Debug)]
pub struct BlockError<T> {
inner: Option<T>,
}
/// This is mostly split out to make the borrow checker happy.
struct Borrow<'a, U: 'a> {
scheduler: &'a mut Scheduler<U>,
num_futures: &'a mut usize,
}
trait SpawnLocal {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>);
}
struct CurrentRunner {
spawn: Cell<Option<*mut SpawnLocal>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
});
/// Run the executor bootstrapping the execution with the provided future.
///
/// This creates a new [`CurrentThread`] executor, spawns the provided future,
/// and blocks the current thread until the provided future and **all**
/// subsequently spawned futures complete. In other words:
///
/// * If the provided bootstrap future does **not** spawn any additional tasks,
/// `block_on_all` returns once `future` completes.
/// * If the provided bootstrap future **does** spawn additional tasks, then
/// `block_on_all` returns once **all** spawned futures complete.
///
/// See [module level][mod] documentation for more details.
///
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
{
let mut current_thread = CurrentThread::new();
let ret = current_thread.block_on(future);
current_thread.run().unwrap();
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
}
/// Executes a future on the current thread.
///
/// The provided future must complete or be canceled before `run` will return.
///
/// Unlike [`tokio::spawn`], this function will always spawn on a
/// `CurrentThread` executor and is able to spawn futures that are not `Send`.
///
/// # Panics
///
/// This function can only be invoked from the context of a `run` call; any
/// other use will result in a panic.
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
{
TaskExecutor::current()
.spawn_local(Box::new(future))
.unwrap();
}
// ===== impl CurrentThread =====
impl CurrentThread<ParkThread> {
/// Create a new instance of `CurrentThread`.
pub fn new() -> Self {
CurrentThread::new_with_park(ParkThread::new())
}
}
impl<P: Park> CurrentThread<P> {
/// Create a new instance of `CurrentThread` backed by the given park
/// handle.
pub fn new_with_park(park: P) -> Self {
let unpark = park.unpark();
let (spawn_sender, spawn_receiver) = mpsc::channel();
let scheduler = Scheduler::new(unpark);
let notify = scheduler.notify();
CurrentThread {
scheduler: scheduler,
num_futures: 0,
park,
spawn_handle: Handle { sender: spawn_sender, notify: notify },
spawn_receiver: spawn_receiver,
}
}
/// Returns `true` if the executor is currently idle.
///
/// An idle executor is defined by not currently having any spawned tasks.
pub fn is_idle(&self) -> bool {
self.num_futures == 0
}
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).turn(duration)
}
/// Bind `CurrentThread` instance with an execution context.
pub fn enter<'a>(&'a mut self, enter: &'a mut Enter) -> Entered<'a, P> {
Entered {
executor: self,
enter,
}
}
/// Returns a reference to the underlying `Park` instance.
pub fn get_park(&self) -> &P {
&self.park
}
/// Returns a mutable reference to the underlying `Park` instance.
pub fn get_park_mut(&mut self) -> &mut P {
&mut self.park
}
fn borrow(&mut self) -> Borrow<P::Unpark> {
Borrow {
scheduler: &mut self.scheduler,
num_futures: &mut self.num_futures,
}
}
/// Get a new handle to spawn futures on the executor
///
/// Different to the executor itself, the handle can be sent to different
/// threads and can be used to spawn futures on the executor.
pub fn handle(&self) -> Handle {
self.spawn_handle.clone()
}
}
impl tokio_executor::Executor for CurrentThread {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.borrow().spawn_local(future);
Ok(())
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, _future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
panic!("Futures 0.2 integration is not available for current_thread");
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures)
.finish()
}
}
// ===== impl Entered =====
impl<'a, P: Park> Entered<'a, P> {
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
match res {
Ok(Async::Ready(e)) => return Ok(e),
Err(e) => return Err(BlockError { inner: Some(e) }),
Ok(Async::NotReady) => {}
}
self.tick();
if let Err(_) = self.executor.park.park() {
return Err(BlockError { inner: None });
}
}
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let res = if self.executor.scheduler.has_pending_futures() {
self.executor.park.park_timeout(Duration::from_millis(0))
} else {
match duration {
Some(duration) => self.executor.park.park_timeout(duration),
None => self.executor.park.park(),
}
};
if res.is_err() {
return Err(TurnError { _p: () });
}
let polled = self.tick();
Ok(Turn { polled })
}
/// Returns a reference to the underlying `Park` instance.
pub fn get_park(&self) -> &P {
&self.executor.park
}
/// Returns a mutable reference to the underlying `Park` instance.
pub fn get_park_mut(&mut self) -> &mut P {
&mut self.executor.park
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
if self.executor.is_idle() {
// Nothing to do
return Ok(());
}
let mut time = dur.map(|dur| (Instant::now() + dur, dur));
loop {
self.tick();
if self.executor.is_idle() {
return Ok(());
}
match time {
Some((until, rem)) => {
if let Err(_) = self.executor.park.park_timeout(rem) {
return Err(RunTimeoutError::new(false));
}
let now = Instant::now();
if now >= until {
return Err(RunTimeoutError::new(true));
}
time = Some((until, until - now));
}
None => {
if let Err(_) = self.executor.park.park() {
return Err(RunTimeoutError::new(false));
}
}
}
}
}
/// Returns `true` if any futures were processed
fn tick(&mut self) -> bool {
// Spawn any futures that were spawned from other threads by manually
// looping over the receiver stream
// FIXME: Slightly ugly but needed to make the borrow checker happy
let (mut borrow, spawn_receiver) = (
Borrow {
scheduler: &mut self.executor.scheduler,
num_futures: &mut self.executor.num_futures,
},
&mut self.executor.spawn_receiver,
);
while let Ok(future) = spawn_receiver.try_recv() {
borrow.spawn_local(future);
}
// After any pending futures were scheduled, do the actual tick
borrow.scheduler.tick(
&mut *self.enter,
borrow.num_futures)
}
}
impl<'a, P: Park> fmt::Debug for Entered<'a, P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Entered")
.field("executor", &self.executor)
.field("enter", &self.enter)
.finish()
}
}
// ===== impl Handle =====
/// Handle to spawn a future on the corresponding `CurrentThread` instance
#[derive(Clone)]
pub struct Handle {
sender: mpsc::Sender<Box<Future<Item = (), Error = ()> + Send + 'static>>,
notify: executor::NotifyHandle,
}
// Manual implementation because the Sender does not implement Debug
impl fmt::Debug for Handle {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Handle")
.finish()
}
}
impl Handle {
/// Spawn a future onto the `CurrentThread` instance corresponding to this handle
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the `CurrentThread`
/// instance of the `Handle` does not exist anymore.
pub fn spawn<F>(&self, future: F) -> Result<(), SpawnError>
where F: Future<Item = (), Error = ()> + Send + 'static {
self.sender.send(Box::new(future))
.expect("CurrentThread does not exist anymore");
// use 0 for the id, CurrentThread does not make use of it
self.notify.notify(0);
Ok(())
}
}
// ===== impl TaskExecutor =====
impl TaskExecutor {
/// Returns an executor that executes futures on the current thread.
///
/// The user of `TaskExecutor` must ensure that when a future is submitted,
/// that it is done within the context of a call to `run`.
///
/// For more details, see the [module level](index.html) documentation.
pub fn current() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
}
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.spawn_local(future)
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, _future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
panic!("Futures 0.2 integration is not available for current_thread");
}
fn status(&self) -> Result<(), SpawnError> {
CURRENT.with(|current| {
if current.spawn.get().is_some() {
Ok(())
} else {
Err(SpawnError::shutdown())
}
})
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future)) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
}
})
}
}
// ===== impl Borrow =====
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.set_spawn(self, || {
f()
})
})
}
}
impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>) {
*self.num_futures += 1;
self.scheduler.schedule(future);
}
}
// ===== impl CurrentRunner =====
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
{
struct Reset<'a>(&'a CurrentRunner);
impl<'a> Drop for Reset<'a> {
fn drop(&mut self) {
self.0.spawn.set(None);
}
}
let _reset = Reset(self);
let spawn = unsafe { hide_lt(spawn as *mut SpawnLocal) };
self.spawn.set(Some(spawn));
f()
}
}
unsafe fn hide_lt<'a>(p: *mut (SpawnLocal + 'a)) -> *mut (SpawnLocal + 'static) {
use std::mem;
mem::transmute(p)
}
// ===== impl RunTimeoutError =====
impl RunTimeoutError {
fn new(timeout: bool) -> Self {
RunTimeoutError { timeout }
}
/// Returns `true` if the error was caused by the operation timing out.
pub fn is_timeout(&self) -> bool {
self.timeout
}
}
impl From<tokio_executor::EnterError> for RunTimeoutError {
fn from(_: tokio_executor::EnterError) -> Self {
RunTimeoutError::new(false)
}
}
// ===== impl BlockError =====
impl<T> BlockError<T> {
/// Returns the error yielded by the future being blocked on
pub fn into_inner(self) -> Option<T> {
self.inner
}
}
impl<T> From<tokio_executor::EnterError> for BlockError<T> {
fn from(_: tokio_executor::EnterError) -> Self {
BlockError { inner: None }
}
}
@@ -52,7 +52,7 @@ struct List<U> {
// Specifically, when a node is stored in at least one of the two lists
// described above, this represents a logical `Arc` handle. This is how
// `Scheduler` maintains its reference to all nodes it manages. Each
// `NotifyHande` instance is an `Arc<Node>` as well.
// `NotifyHandle` instance is an `Arc<Node>` as well.
//
// When `Scheduler` drops, it clears the linked list of all nodes that it
// manages. When doing so, it must attempt to decrement the reference count (by
@@ -196,6 +196,15 @@ where U: Unpark,
self.inner.enqueue(ptr);
}
/// Returns `true` if there are currently any pending futures
pub fn has_pending_futures(&mut self) -> bool {
// See function definition for why the unsafe is needed and
// correctly used here
unsafe {
self.inner.has_pending_futures()
}
}
/// Advance the scheduler state, returning `true` if any futures were
/// processed.
///
@@ -439,6 +448,22 @@ impl<U> Inner<U> {
}
}
/// Returns `true` if there are currently any pending futures
///
/// See `dequeue` for an explanation why this function is unsafe.
unsafe fn has_pending_futures(&self) -> bool {
let tail = *self.tail_readiness.get();
let next = (*tail).next_readiness.load(Acquire);
if tail == self.stub() {
if next.is_null() {
return false;
}
}
true
}
/// The dequeue function from the 1024cores intrusive MPSC queue algorithm
///
/// Note that this unsafe as it required mutual exclusion (only one thread
@@ -617,7 +642,7 @@ impl<'a, U> Clone for Notify<'a, U> {
impl<'a, U> fmt::Debug for Notify<'a, U> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Notiy").finish()
fmt.debug_struct("Notify").finish()
}
}
@@ -0,0 +1,622 @@
#![cfg(not(feature = "unstable-futures"))]
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate futures;
use tokio_current_thread::{block_on_all, CurrentThread};
use std::any::Any;
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::prelude::*;
use futures::sync::oneshot;
#[test]
fn spawn_from_block_on_all() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let msg = tokio_current_thread::block_on_all(lazy(move || {
c.set(1 + c.get());
// Spawn!
tokio_current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Rc::new(Cell::new(0));
let cnt2 = cnt.clone();
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
assert_eq!(1, cnt2.get());
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Rc::new(Cell::new(0));
let mut tokio_current_thread = CurrentThread::new();
for _ in 0..ITER {
let cnt = cnt.clone();
tokio_current_thread.spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok::<(), ()>(())
}));
}
tokio_current_thread.run().unwrap();
assert_eq!(cnt.get(), ITER);
}
#[test]
fn does_not_set_global_executor_by_default() {
use tokio_executor::Executor;
block_on_all(lazy(|| {
tokio_executor::DefaultExecutor::current()
.spawn(Box::new(lazy(|| ok())))
.unwrap_err();
ok()
})).unwrap();
}
#[test]
fn spawn_from_block_on_future() {
let cnt = Rc::new(Cell::new(0));
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
tokio_current_thread::spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
}));
Ok::<_, ()>(())
})).unwrap();
tokio_current_thread.run().unwrap();
assert_eq!(1, cnt.get());
}
struct Never(Rc<()>);
impl Future for Never {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(Async::NotReady)
}
}
#[test]
fn outstanding_tasks_are_dropped_when_executor_is_dropped() {
let mut rc = Rc::new(());
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.spawn(Never(rc.clone()));
drop(tokio_current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
// Using the global spawn fn
let mut rc = Rc::new(());
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
tokio_current_thread::spawn(Never(rc.clone()));
Ok::<_, ()>(())
})).unwrap();
drop(tokio_current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
}
#[test]
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
})).unwrap();
}
#[test]
#[should_panic]
fn run_in_future() {
block_on_all(lazy(|| {
tokio_current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
}));
ok()
})).unwrap();
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
struct Infini {
num: Rc<Cell<usize>>,
}
impl Future for Infini {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.num.set(1 + self.num.get());
task::current().notify();
Ok(Async::NotReady)
}
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.turn(None)
.unwrap();
assert_eq!(1, num.get());
}
#[test]
fn tasks_are_scheduled_fairly() {
let state = Rc::new(RefCell::new([0, 0]));
struct Spin {
state: Rc<RefCell<[i32; 2]>>,
idx: usize,
}
impl Future for Spin {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
let mut state = self.state.borrow_mut();
if self.idx == 0 {
let diff = state[0] - state[1];
assert!(diff.abs() <= 1);
if state[0] >= 50 {
return Ok(().into());
}
}
state[self.idx] += 1;
if state[self.idx] >= 100 {
return Ok(().into());
}
task::current().notify();
Ok(Async::NotReady)
}
}
block_on_all(lazy(|| {
tokio_current_thread::spawn(Spin {
state: state.clone(),
idx: 0,
});
tokio_current_thread::spawn(Spin {
state: state,
idx: 1,
});
ok()
})).unwrap();
}
#[test]
fn spawn_and_turn() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let mut tokio_current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
tokio_current_thread.spawn(lazy(move || {
Ok(())
}));
// Turn once...
tokio_current_thread.turn(None).unwrap();
tokio_current_thread.spawn(lazy(move || {
c.set(1 + c.get());
// Spawn!
tokio_current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok(())
}));
// This does not run the newly spawned thread
tokio_current_thread.turn(None).unwrap();
assert_eq!(1, cnt.get());
// This runs the newly spawned thread
tokio_current_thread.turn(None).unwrap();
assert_eq!(2, cnt.get());
}
#[test]
fn spawn_in_drop() {
let mut tokio_current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
tokio_current_thread.spawn({
struct OnDrop<F: FnOnce()>(Option<F>);
impl<F: FnOnce()> Drop for OnDrop<F> {
fn drop(&mut self) {
(self.0.take().unwrap())();
}
}
struct MyFuture {
_data: Box<Any>,
}
impl Future for MyFuture {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(().into())
}
}
MyFuture {
_data: Box::new(OnDrop(Some(move || {
tokio_current_thread::spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}));
}))),
}
});
tokio_current_thread.block_on(rx).unwrap();
tokio_current_thread.run().unwrap();
}
#[test]
fn hammer_turn() {
use futures::sync::mpsc;
const ITER: usize = 100;
const N: usize = 100;
const THREADS: usize = 4;
for _ in 0..ITER {
let mut ths = vec![];
// Add some jitter
for _ in 0..THREADS {
let th = thread::spawn(|| {
let mut tokio_current_thread = CurrentThread::new();
let (tx, rx) = mpsc::unbounded();
tokio_current_thread.spawn({
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
rx.for_each(move |_| {
c.set(1 + c.get());
Ok(())
})
.map_err(|e| panic!("err={:?}", e))
.map(move |v| {
assert_eq!(N, cnt.get());
v
})
});
thread::spawn(move || {
for _ in 0..N {
tx.unbounded_send(()).unwrap();
thread::yield_now();
}
});
while !tokio_current_thread.is_idle() {
tokio_current_thread.turn(None).unwrap();
}
});
ths.push(th);
}
for th in ths {
th.join().unwrap();
}
}
}
#[test]
fn turn_has_polled() {
let mut tokio_current_thread = CurrentThread::new();
// Spawn oneshot receiver
let (sender, receiver) = oneshot::channel::<()>();
tokio_current_thread.spawn(receiver.then(|_| Ok(())));
// Turn once...
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// Should've polled the receiver once, but considered it not ready
assert!(res.has_polled());
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// Should've polled nothing, the receiver is not ready yet
assert!(!res.has_polled());
// Make the receiver ready
sender.send(()).unwrap();
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// Should've polled the receiver, it's ready now
assert!(res.has_polled());
// Now the executor should be empty
assert!(tokio_current_thread.is_idle());
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// So should've polled nothing
assert!(!res.has_polled());
}
// Our own mock Park that is never really waiting and the only
// thing it does is to send, on request, something (once) to a onshot
// channel
struct MyPark {
sender: Option<oneshot::Sender<()>>,
send_now: Rc<Cell<bool>>,
}
struct MyUnpark;
impl tokio_executor::park::Park for MyPark {
type Unpark = MyUnpark;
type Error = ();
fn unpark(&self) -> Self::Unpark {
MyUnpark
}
fn park(&mut self) -> Result<(), Self::Error> {
// If called twice with send_now, this will intentionally panic
if self.send_now.get() {
self.sender.take().unwrap().send(()).unwrap();
}
Ok(())
}
fn park_timeout(&mut self, _duration: Duration) -> Result<(), Self::Error> {
self.park()
}
}
impl tokio_executor::park::Unpark for MyUnpark {
fn unpark(&self) {}
}
#[test]
fn turn_fair() {
let send_now = Rc::new(Cell::new(false));
let (sender, receiver) = oneshot::channel::<()>();
let (sender_2, receiver_2) = oneshot::channel::<()>();
let (sender_3, receiver_3) = oneshot::channel::<()>();
let my_park = MyPark {
sender: Some(sender_3),
send_now: send_now.clone(),
};
let mut tokio_current_thread = CurrentThread::new_with_park(my_park);
let receiver_1_done = Rc::new(Cell::new(false));
let receiver_1_done_clone = receiver_1_done.clone();
// Once an item is received on the oneshot channel, it will immediately
// immediately make the second oneshot channel ready
tokio_current_thread.spawn(receiver
.map_err(|_| unreachable!())
.and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
Ok(())
})
);
let receiver_2_done = Rc::new(Cell::new(false));
let receiver_2_done_clone = receiver_2_done.clone();
tokio_current_thread.spawn(receiver_2
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
})
);
// The third receiver is only woken up from our Park implementation, it simulates
// e.g. a socket that first has to be polled to know if it is ready now
let receiver_3_done = Rc::new(Cell::new(false));
let receiver_3_done_clone = receiver_3_done.clone();
tokio_current_thread.spawn(receiver_3
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
})
);
// First turn should've polled both and considered them not ready
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
assert!(res.has_polled());
// Next turn should've polled nothing
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
assert!(!res.has_polled());
assert!(!receiver_1_done.get());
assert!(!receiver_2_done.get());
assert!(!receiver_3_done.get());
// After this the receiver future will wake up the second receiver future,
// so there are pending futures again
sender.send(()).unwrap();
// Now the first receiver should be done, the second receiver should be ready
// to be polled again and the socket not yet
let res = tokio_current_thread.turn(None).unwrap();
assert!(res.has_polled());
assert!(receiver_1_done.get());
assert!(!receiver_2_done.get());
assert!(!receiver_3_done.get());
// Now let our park implementation know that it should send something to sender 3
send_now.set(true);
// This should resolve the second receiver directly, but also poll the socket
// and read the packet from it. If it didn't do both here, we would handle
// futures that are woken up from the reactor and directly unfairly and would
// favour the ones that are woken up directly.
let res = tokio_current_thread.turn(None).unwrap();
assert!(res.has_polled());
assert!(receiver_1_done.get());
assert!(receiver_2_done.get());
assert!(receiver_3_done.get());
// Don't send again
send_now.set(false);
// Now we should be idle and turning should not poll anything
assert!(tokio_current_thread.is_idle());
let res = tokio_current_thread.turn(None).unwrap();
assert!(!res.has_polled());
}
#[test]
fn spawn_from_other_thread() {
let mut current_thread = CurrentThread::new();
let handle = current_thread.handle();
let (sender, receiver) = oneshot::channel::<()>();
thread::spawn(move || {
handle.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
})).unwrap();
});
let _ = current_thread.block_on(receiver).unwrap();
}
#[test]
fn spawn_from_other_thread_unpark() {
use std::sync::mpsc::channel as mpsc_channel;
let mut current_thread = CurrentThread::new();
let handle = current_thread.handle();
let (sender_1, receiver_1) = oneshot::channel::<()>();
let (sender_2, receiver_2) = mpsc_channel::<()>();
thread::spawn(move || {
let _ = receiver_2.recv().unwrap();
handle.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
})).unwrap();
});
// Ensure that unparking the executor works correctly. It will first
// check if there are new futures (there are none), then execute the
// lazy future below which will cause the future to be spawned from
// the other thread. Then the executor will park but should be woken
// up because *now* we have a new future to schedule
let _ = current_thread.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1)
).unwrap();
}
fn ok() -> future::FutureResult<(), ()> {
future::ok(())
}
+8
View File
@@ -1,3 +1,11 @@
# 0.1.2 (March 30, 2018)
* Implement `Unpark` for `Box<Unpark>`.
# 0.1.1 (March 22, 2018)
* Optionally support futures 0.2.
# 0.1.0 (March 09, 2018)
* Initial release
+8 -3
View File
@@ -1,10 +1,15 @@
[package]
name = "tokio-executor"
version = "0.1.0"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.2"
documentation = "https://docs.rs/tokio-executor"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT/Apache-2.0"
license = "MIT"
authors = ["Carl Lerche <[email protected]>"]
description = """
Future execution primitives
@@ -13,4 +18,4 @@ keywords = ["futures", "tokio"]
categories = ["concurrency", "asynchronous"]
[dependencies]
futures = "0.1.18"
futures = "0.1.19"
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2018 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+3 -10
View File
@@ -38,17 +38,10 @@ executor, including:
## License
This project is licensed under either of
* Apache License, Version 2.0, ([LICENSE-APACHE](../LICENSE-APACHE) or
http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](../LICENSE-MIT) or
http://opensource.org/licenses/MIT)
at your option.
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+9
View File
@@ -2,6 +2,9 @@ use std::prelude::v1::*;
use std::cell::Cell;
use std::fmt;
#[cfg(feature = "unstable-futures")]
use futures2;
thread_local!(static ENTERED: Cell<bool> = Cell::new(false));
/// Represents an executor context.
@@ -10,6 +13,9 @@ thread_local!(static ENTERED: Cell<bool> = Cell::new(false));
pub struct Enter {
on_exit: Vec<Box<Callback>>,
permanent: bool,
#[cfg(feature = "unstable-futures")]
_enter2: futures2::executor::Enter,
}
/// An error returned by `enter` if an execution scope has already been
@@ -40,6 +46,9 @@ pub fn enter() -> Result<Enter, EnterError> {
Ok(Enter {
on_exit: Vec::new(),
permanent: false,
#[cfg(feature = "unstable-futures")]
_enter2: futures2::executor::enter().unwrap(),
})
}
})
+30 -1
View File
@@ -6,6 +6,9 @@ use std::cell::Cell;
use std::marker::PhantomData;
use std::rc::Rc;
#[cfg(feature = "unstable-futures")]
use futures2;
/// Executes futures on the default executor for the current execution context.
///
/// `DefaultExecutor` implements `Executor` and can be used to spawn futures
@@ -28,7 +31,7 @@ impl DefaultExecutor {
/// Futures may be spawned onto the default executor using this handle.
///
/// The returned handle will reference whichever executor is configured as
/// the default **at the time `spawn` is called`. This enables
/// the default **at the time `spawn` is called**. This enables
/// `DefaultExecutor::current()` to be called before an execution context is
/// setup, then passed **into** an execution context before it is used.
pub fn current() -> DefaultExecutor {
@@ -59,6 +62,23 @@ impl super::Executor for DefaultExecutor {
}
})
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
EXECUTOR.with(|current_executor| {
match current_executor.get() {
Some(executor) => {
let executor = unsafe { &mut *executor };
executor.spawn2(future)
}
None => {
Err(futures2::executor::SpawnError::shutdown())
}
}
})
}
}
// ===== global spawn fns =====
@@ -109,6 +129,15 @@ pub fn spawn<T>(future: T)
.unwrap()
}
/// Like `spawn` but compatible with futures 0.2
#[cfg(feature = "unstable-futures")]
pub fn spawn2<T>(future: T)
where T: futures2::Future<Item = (), Error = futures2::Never> + Send + 'static,
{
DefaultExecutor::current().spawn2(Box::new(future))
.unwrap()
}
/// Set the default executor for the duration of the closure
///
/// # Panics
+13 -2
View File
@@ -31,10 +31,13 @@
//! [`Park`]: park/index.html
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.0")]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.2")]
extern crate futures;
#[cfg(feature = "unstable-futures")]
extern crate futures2;
mod enter;
mod global;
pub mod park;
@@ -42,6 +45,9 @@ pub mod park;
pub use enter::{enter, Enter, EnterError};
pub use global::{spawn, with_default, DefaultExecutor};
#[cfg(feature = "unstable-futures")]
pub use global::spawn2;
use futures::Future;
/// A value that executes futures.
@@ -129,7 +135,12 @@ pub trait Executor {
/// # fn main() {}
/// ```
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>;
-> Result<(), SpawnError>;
/// Like `spawn`, but compatible with futures 0.2
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
+11 -5
View File
@@ -29,7 +29,7 @@
//!
//! * If [`unpark`] is called before [`park`], the next call to [`park`] will
//! **not** block the thread.
//! * **Spurious** wakeups are permited, i.e., the [`park`] method may unblock
//! * **Spurious** wakeups are permitted, i.e., the [`park`] method may unblock
//! even if [`unpark`] was not called.
//! * [`park_timeout`] does the same as [`park`] but allows specifying a maximum
//! time to block the thread for.
@@ -75,7 +75,7 @@ pub trait Park {
///
/// # Panics
///
/// This function **should** not panic, but ultimiately, panics are left as
/// This function **should** not panic, but ultimately, panics are left as
/// an implementation detail. Refer to the documentation for the specific
/// `Park` implementation
///
@@ -95,7 +95,7 @@ pub trait Park {
///
/// # Panics
///
/// This function **should** not panic, but ultimiately, panics are left as
/// This function **should** not panic, but ultimately, panics are left as
/// an implementation detail. Refer to the documentation for the specific
/// `Park` implementation
///
@@ -119,7 +119,7 @@ pub trait Unpark: Sync + Send + 'static {
///
/// # Panics
///
/// This function **should** not panic, but ultimiately, panics are left as
/// This function **should** not panic, but ultimately, panics are left as
/// an implementation detail. Refer to the documentation for the specific
/// `Unpark` implementation
///
@@ -127,6 +127,12 @@ pub trait Unpark: Sync + Send + 'static {
fn unpark(&self);
}
impl Unpark for Box<Unpark> {
fn unpark(&self) {
(**self).unpark()
}
}
/// Blocks the current thread using a condition variable.
///
/// Implements the [`Park`] functionality by using a condition variable. An
@@ -258,7 +264,7 @@ impl Inner {
None => self.condvar.wait(m).unwrap(),
};
// Transition back to idle. If the state has transitione dto `NOTIFY`,
// Transition back to idle. If the state has transitioned to `NOTIFY`,
// this will consume that notification
self.state.store(IDLE, Ordering::SeqCst);
+9
View File
@@ -0,0 +1,9 @@
# 0.1.1 (June 13, 2018)
* Add `OpenOptions` (#390)
* Add `into_std` to `File` (#403)
* Use `tokio-codec` in examples
# 0.1.0 (May 2, 2018)
* Initial release
+30
View File
@@ -0,0 +1,30 @@
[package]
name = "tokio-fs"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.1"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-fs/0.1"
description = """
Filesystem API for Tokio.
"""
keywords = ["tokio", "futures", "fs", "file", "async"]
categories = ["asynchronous", "network-programming", "filesystem"]
[dependencies]
futures = "0.1.21"
tokio-threadpool = { version = "0.1.3", path = "../tokio-threadpool" }
tokio-io = { version = "0.1.6", path = "../tokio-io" }
[dev-dependencies]
rand = "0.4.2"
tempdir = "0.3.7"
tokio-io = { version = "0.1.6", path = "../tokio-io" }
tokio-codec = { version = "0.1.0", path = "../tokio-codec" }
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2018 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+19
View File
@@ -0,0 +1,19 @@
# Tokio FS
Asynchronous filesystem manipulation operations (and stdin, stdout, stderr).
[Documentation](https://tokio-rs.github.io/tokio/tokio_fs/)
## Overview
This crate provides filesystem manipulation facilities for usage with Tokio.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+48
View File
@@ -0,0 +1,48 @@
//! Echo everything received on STDIN to STDOUT.
#![deny(deprecated, warnings)]
extern crate futures;
extern crate tokio_fs;
extern crate tokio_codec;
extern crate tokio_threadpool;
use tokio_fs::{stdin, stdout, stderr};
use tokio_codec::{FramedRead, FramedWrite, LinesCodec};
use tokio_threadpool::Builder;
use futures::{Future, Stream, Sink};
use std::io;
pub fn main() {
let pool = Builder::new()
.pool_size(1)
.build();
pool.spawn({
let input = FramedRead::new(stdin(), LinesCodec::new());
let output = FramedWrite::new(stdout(), LinesCodec::new())
.with(|line: String| {
let mut out = "OUT: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let error = FramedWrite::new(stderr(), LinesCodec::new())
.with(|line: String| {
let mut out = "ERR: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let dst = output.fanout(error);
input
.forward(dst)
.map(|_| ())
.map_err(|e| panic!("io error = {:?}", e))
});
pool.shutdown_on_idle().wait().unwrap();
}
+37
View File
@@ -0,0 +1,37 @@
use super::File;
use futures::{Future, Poll};
use std::fs::File as StdFile;
use std::io;
use std::path::Path;
/// Future returned by `File::create` and resolves to a `File` instance.
#[derive(Debug)]
pub struct CreateFuture<P> {
path: P,
}
impl<P> CreateFuture<P>
where P: AsRef<Path> + Send + 'static,
{
pub(crate) fn new(path: P) -> Self {
CreateFuture { path }
}
}
impl<P> Future for CreateFuture<P>
where P: AsRef<Path> + Send + 'static,
{
type Item = File;
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
let std = try_ready!(::blocking_io(|| {
StdFile::create(&self.path)
}));
let file = File::from_std(std);
Ok(file.into())
}
}
+222
View File
@@ -0,0 +1,222 @@
//! Types for working with [`File`].
//!
//! [`File`]: file/struct.File.html
mod create;
mod open;
mod open_options;
pub use self::create::CreateFuture;
pub use self::open::OpenFuture;
pub use self::open_options::OpenOptions;
use tokio_io::{AsyncRead, AsyncWrite};
use futures::Poll;
use std::fs::{File as StdFile, Metadata, Permissions};
use std::io::{self, Read, Write, Seek};
use std::path::Path;
/// A reference to an open file on the filesystem.
///
/// This is a specialized version of [`std::fs::File`][std] for usage from the
/// Tokio runtime.
///
/// An instance of a `File` can be read and/or written depending on what options
/// it was opened with. Files also implement Seek to alter the logical cursor
/// that the file contains internally.
///
/// Files are automatically closed when they go out of scope.
///
/// [std]: https://doc.rust-lang.org/std/fs/struct.File.html
#[derive(Debug)]
pub struct File {
std: Option<StdFile>,
}
impl File {
/// Attempts to open a file in read-only mode.
///
/// See [`OpenOptions`] for more details.
///
/// [`OpenOptions`]: struct.OpenOptions.html
///
/// # Errors
///
/// `OpenFuture` results in an error if called from outside of the Tokio
/// runtime or if the underlying [`open`] call results in an error.
///
/// [`open`]: https://doc.rust-lang.org/std/fs/struct.File.html#method.open
pub fn open<P>(path: P) -> OpenFuture<P>
where P: AsRef<Path> + Send + 'static,
{
OpenOptions::new().read(true).open(path)
}
/// Opens a file in write-only mode.
///
/// This function will create a file if it does not exist, and will truncate
/// it if it does.
///
/// See [`OpenOptions`] for more details.
///
/// [`OpenOptions`]: struct.OpenOptions.html
///
/// # Errors
///
/// `CreateFuture` results in an error if called from outside of the Tokio
/// runtime or if the underlying [`create`] call results in an error.
///
/// [`create`]: https://doc.rust-lang.org/std/fs/struct.File.html#method.create
pub fn create<P>(path: P) -> CreateFuture<P>
where P: AsRef<Path> + Send + 'static,
{
CreateFuture::new(path)
}
/// Convert a [`std::fs::File`][std] to a `tokio_fs::File`.
///
/// [std]: https://doc.rust-lang.org/std/fs/struct.File.html
pub(crate) fn from_std(std: StdFile) -> File {
File { std: Some(std) }
}
/// Seek to an offset, in bytes, in a stream.
///
/// A seek beyond the end of a stream is allowed, but implementation
/// defined.
///
/// If the seek operation completed successfully, this method returns the
/// new position from the start of the stream. That position can be used
/// later with `SeekFrom::Start`.
///
/// # Errors
///
/// Seeking to a negative offset is considered an error.
pub fn poll_seek(&mut self, pos: io::SeekFrom) -> Poll<u64, io::Error> {
::blocking_io(|| self.std().seek(pos))
}
/// Attempts to sync all OS-internal metadata to disk.
///
/// This function will attempt to ensure that all in-core data reaches the
/// filesystem before returning.
pub fn poll_sync_all(&mut self) -> Poll<(), io::Error> {
::blocking_io(|| self.std().sync_all())
}
/// This function is similar to `poll_sync_all`, except that it may not
/// synchronize file metadata to the filesystem.
///
/// This is intended for use cases that must synchronize content, but don't
/// need the metadata on disk. The goal of this method is to reduce disk
/// operations.
///
/// Note that some platforms may simply implement this in terms of `poll_sync_all`.
pub fn poll_sync_data(&mut self) -> Poll<(), io::Error> {
::blocking_io(|| self.std().sync_data())
}
/// Truncates or extends the underlying file, updating the size of this file to become size.
///
/// If the size is less than the current file's size, then the file will be
/// shrunk. If it is greater than the current file's size, then the file
/// will be extended to size and have all of the intermediate data filled in
/// with 0s.
///
/// # Errors
///
/// This function will return an error if the file is not opened for
/// writing.
pub fn poll_set_len(&mut self, size: u64) -> Poll<(), io::Error> {
::blocking_io(|| self.std().set_len(size))
}
/// Queries metadata about the underlying file.
pub fn poll_metadata(&mut self) -> Poll<Metadata, io::Error> {
::blocking_io(|| self.std().metadata())
}
/// Create a new `File` instance that shares the same underlying file handle
/// as the existing `File` instance. Reads, writes, and seeks will affect both
/// File instances simultaneously.
pub fn poll_try_clone(&mut self) -> Poll<File, io::Error> {
::blocking_io(|| {
let std = self.std().try_clone()?;
Ok(File::from_std(std))
})
}
/// Changes the permissions on the underlying file.
///
/// # Platform-specific behavior
///
/// This function currently corresponds to the `fchmod` function on Unix and
/// the `SetFileInformationByHandle` function on Windows. Note that, this
/// [may change in the future][changes].
///
/// [changes]: https://doc.rust-lang.org/std/io/index.html#platform-specific-behavior
///
/// # Errors
///
/// This function will return an error if the user lacks permission change
/// attributes on the underlying file. It may also return an error in other
/// os-specific unspecified cases.
pub fn poll_set_permissions(&mut self, perm: Permissions) -> Poll<(), io::Error> {
::blocking_io(|| self.std().set_permissions(perm))
}
/// Destructures the `tokio_fs::File` into a [`std::fs::File`][std].
///
/// # Panics
///
/// This function will panic if [`shutdown`] has been called.
pub fn into_std(mut self) -> StdFile {
self.std.take().expect("`File` instance already shutdown")
}
fn std(&mut self) -> &mut StdFile {
self.std.as_mut().expect("`File` instance already shutdown")
}
}
impl Read for File {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
::would_block(|| self.std().read(buf))
}
}
impl AsyncRead for File {
unsafe fn prepare_uninitialized_buffer(&self, _: &mut [u8]) -> bool {
false
}
}
impl Write for File {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
::would_block(|| self.std().write(buf))
}
fn flush(&mut self) -> io::Result<()> {
::would_block(|| self.std().flush())
}
}
impl AsyncWrite for File {
fn shutdown(&mut self) -> Poll<(), io::Error> {
::blocking_io(|| {
self.std = None;
Ok(())
})
}
}
impl Drop for File {
fn drop(&mut self) {
if let Some(_std) = self.std.take() {
// This is probably fine as closing a file *shouldn't* be a blocking
// operation. That said, ideally `shutdown` is called first.
}
}
}
+38
View File
@@ -0,0 +1,38 @@
use super::File;
use futures::{Future, Poll};
use std::fs::OpenOptions as StdOpenOptions;
use std::io;
use std::path::Path;
/// Future returned by `File::open` and resolves to a `File` instance.
#[derive(Debug)]
pub struct OpenFuture<P> {
options: StdOpenOptions,
path: P,
}
impl<P> OpenFuture<P>
where P: AsRef<Path> + Send + 'static,
{
pub(crate) fn new(options: StdOpenOptions, path: P) -> Self {
OpenFuture { options, path }
}
}
impl<P> Future for OpenFuture<P>
where P: AsRef<Path> + Send + 'static,
{
type Item = File;
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
let std = try_ready!(::blocking_io(|| {
self.options.open(&self.path)
}));
let file = File::from_std(std);
Ok(file.into())
}
}
+103
View File
@@ -0,0 +1,103 @@
use super::OpenFuture;
use std::convert::From;
use std::fs::OpenOptions as StdOpenOptions;
use std::path::Path;
/// Options and flags which can be used to configure how a file is opened.
///
/// This is a specialized version of [`std::fs::OpenOptions`] for usage from
/// the Tokio runtime.
///
/// `From<std::fs::OpenOptions>` is implemented for more advanced configuration
/// than the methods provided here.
///
/// [`std::fs::OpenOptions`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html
#[derive(Clone, Debug)]
pub struct OpenOptions(StdOpenOptions);
impl OpenOptions {
/// Creates a blank new set of options ready for configuration.
///
/// All options are initially set to `false`.
///
/// # Examples
///
/// ```ignore
/// use tokio::fs::OpenOptions;
///
/// let mut options = OpenOptions::new();
/// let future = options.read(true).open("foo.txt");
/// ```
pub fn new() -> OpenOptions {
OpenOptions(StdOpenOptions::new())
}
/// See the underlying [`read`] call for details.
///
/// [`read`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.read
pub fn read(&mut self, read: bool) -> &mut OpenOptions {
self.0.read(read);
self
}
/// See the underlying [`write`] call for details.
///
/// [`write`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.write
pub fn write(&mut self, write: bool) -> &mut OpenOptions {
self.0.write(write);
self
}
/// See the underlying [`append`] call for details.
///
/// [`append`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.append
pub fn append(&mut self, append: bool) -> &mut OpenOptions {
self.0.append(append);
self
}
/// See the underlying [`truncate`] call for details.
///
/// [`truncate`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.truncate
pub fn truncate(&mut self, truncate: bool) -> &mut OpenOptions {
self.0.truncate(truncate);
self
}
/// See the underlying [`create`] call for details.
///
/// [`create`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.create
pub fn create(&mut self, create: bool) -> &mut OpenOptions {
self.0.create(create);
self
}
/// See the underlying [`create_new`] call for details.
///
/// [`create_new`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.create_new
pub fn create_new(&mut self, create_new: bool) -> &mut OpenOptions {
self.0.create_new(create_new);
self
}
/// Opens a file at `path` with the options specified by `self`.
///
/// # Errors
///
/// `OpenOptionsFuture` results in an error if called from outside of the
/// Tokio runtime or if the underlying [`open`] call results in an error.
///
/// [`open`]: https://doc.rust-lang.org/std/fs/struct.OpenOptions.html#method.open
pub fn open<P>(&self, path: P) -> OpenFuture<P>
where P: AsRef<Path> + Send + 'static
{
OpenFuture::new(self.0.clone(), path)
}
}
impl From<StdOpenOptions> for OpenOptions {
fn from(options: StdOpenOptions) -> OpenOptions {
OpenOptions(options)
}
}
+68
View File
@@ -0,0 +1,68 @@
//! Asynchronous filesystem manipulation operations (and stdin, stdout, stderr).
//!
//! This module contains basic methods and types for manipulating the contents
//! of the local filesystem from within the context of the Tokio runtime.
//!
//! Tasks running on the Tokio runtime are expected to be asynchronous, i.e.,
//! they will not block the thread of execution. Filesystem operations do not
//! satisfy this requirement. In order to perform filesystem operations
//! asynchronously, this library uses the [`blocking`][blocking] annotation
//! to signal to the runtime that a blocking operation is being performed. This
//! allows the runtime to compensate.
//!
//! [blocking]: https://docs.rs/tokio-threadpool/0.1/tokio_threadpool/fn.blocking.html
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-fs/0.1.1")]
#[macro_use]
extern crate futures;
extern crate tokio_io;
extern crate tokio_threadpool;
pub mod file;
mod stdin;
mod stdout;
mod stderr;
pub use file::File;
pub use file::OpenOptions;
pub use stdin::{stdin, Stdin};
pub use stdout::{stdout, Stdout};
pub use stderr::{stderr, Stderr};
use futures::Poll;
use futures::Async::*;
use std::io;
use std::io::ErrorKind::{Other, WouldBlock};
fn blocking_io<F, T>(f: F) -> Poll<T, io::Error>
where F: FnOnce() -> io::Result<T>,
{
match tokio_threadpool::blocking(f) {
Ok(Ready(Ok(v))) => Ok(v.into()),
Ok(Ready(Err(err))) => Err(err),
Ok(NotReady) => Ok(NotReady),
Err(_) => Err(blocking_err()),
}
}
fn would_block<F, T>(f: F) -> io::Result<T>
where F: FnOnce() -> io::Result<T>,
{
match tokio_threadpool::blocking(f) {
Ok(Ready(Ok(v))) => Ok(v),
Ok(Ready(Err(err))) => {
debug_assert_ne!(err.kind(), WouldBlock);
Err(err)
}
Ok(NotReady) => Err(WouldBlock.into()),
Err(_) => Err(blocking_err()),
}
}
fn blocking_err() -> io::Error {
io::Error::new(Other, "`blocking` annotated I/O must be called \
from the context of the Tokio runtime.")
}
+45
View File
@@ -0,0 +1,45 @@
use tokio_io::{AsyncWrite};
use futures::Poll;
use std::io::{self, Write, Stderr as StdStderr};
/// A handle to the standard error stream of a process.
///
/// The handle implements the [`AsyncWrite`] trait, but beware that concurrent
/// writes to `Stderr` must be executed with care.
///
/// Created by the [`stderr`] function.
///
/// [`stderr`]: fn.stderr.html
/// [`AsyncWrite`]: trait.AsyncWrite.html
#[derive(Debug)]
pub struct Stderr {
std: StdStderr,
}
/// Constructs a new handle to the standard error of the current process.
///
/// The returned handle allows writing to standard error from the within the
/// Tokio runtime.
pub fn stderr() -> Stderr {
let std = io::stderr();
Stderr { std }
}
impl Write for Stderr {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
::would_block(|| self.std.write(buf))
}
fn flush(&mut self) -> io::Result<()> {
::would_block(|| self.std.flush())
}
}
impl AsyncWrite for Stderr {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
+38
View File
@@ -0,0 +1,38 @@
use tokio_io::{AsyncRead};
use std::io::{self, Read, Stdin as StdStdin};
/// A handle to the standard input stream of a process.
///
/// The handle implements the [`AsyncRead`] trait, but beware that concurrent
/// reads of `Stdin` must be executed with care.
///
/// Created by the [`stdin`] function.
///
/// [`stdin`]: fn.stdin.html
/// [`AsyncRead`]: trait.AsyncRead.html
#[derive(Debug)]
pub struct Stdin {
std: StdStdin,
}
/// Constructs a new handle to the standard input of the current process.
///
/// The returned handle allows reading from standard input from the within the
/// Tokio runtime.
pub fn stdin() -> Stdin {
let std = io::stdin();
Stdin { std }
}
impl Read for Stdin {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
::would_block(|| self.std.read(buf))
}
}
impl AsyncRead for Stdin {
unsafe fn prepare_uninitialized_buffer(&self, _: &mut [u8]) -> bool {
false
}
}
+44
View File
@@ -0,0 +1,44 @@
use tokio_io::{AsyncWrite};
use futures::Poll;
use std::io::{self, Write, Stdout as StdStdout};
/// A handle to the standard output stream of a process.
///
/// The handle implements the [`AsyncWrite`] trait, but beware that concurrent
/// writes to `Stdout` must be executed with care.
///
/// Created by the [`stdout`] function.
///
/// [`stdout`]: fn.stdout.html
/// [`AsyncWrite`]: trait.AsyncWrite.html
#[derive(Debug)]
pub struct Stdout {
std: StdStdout,
}
/// Constructs a new handle to the standard output of the current process.
///
/// The returned handle allows writing to standard out from the within the Tokio
/// runtime.
pub fn stdout() -> Stdout {
let std = io::stdout();
Stdout { std }
}
impl Write for Stdout {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
::would_block(|| self.std.write(buf))
}
fn flush(&mut self) -> io::Result<()> {
::would_block(|| self.std.flush())
}
}
impl AsyncWrite for Stdout {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
+73
View File
@@ -0,0 +1,73 @@
extern crate futures;
extern crate rand;
extern crate tempdir;
extern crate tokio_fs;
extern crate tokio_io;
extern crate tokio_threadpool;
use tokio_fs::*;
use tokio_io::io;
use tokio_threadpool::*;
use futures::Future;
use futures::future::poll_fn;
use futures::sync::oneshot;
use rand::{thread_rng, Rng};
use tempdir::TempDir;
use std::fs::File as StdFile;
use std::io::Read;
#[test]
fn read_write() {
const NUM_CHARS: usize = 16 * 1_024;
let dir = TempDir::new("tokio-fs-tests").unwrap();
let file_path = dir.path().join("read_write.txt");
let contents: Vec<u8> = thread_rng().gen_ascii_chars()
.take(NUM_CHARS)
.collect::<String>()
.into();
let pool = Builder::new()
.pool_size(1)
.build();
let (tx, rx) = oneshot::channel();
pool.spawn({
let file_path = file_path.clone();
let contents = contents.clone();
File::create(file_path)
.and_then(move |file| io::write_all(file, contents))
.and_then(|(mut file, _)| {
poll_fn(move || file.poll_sync_all())
})
.then(|res| {
let _ = res.unwrap();
tx.send(()).unwrap();
Ok(())
})
});
rx.wait().unwrap();
let mut file = StdFile::open(&file_path).unwrap();
let mut dst = vec![];
file.read_to_end(&mut dst).unwrap();
assert_eq!(dst, contents);
pool.spawn({
File::open(file_path)
.and_then(|file| io::read_to_end(file, vec![]))
.then(move |res| {
let (_, buf) = res.unwrap();
assert_eq!(buf, contents);
Ok(())
})
});
}
+4
View File
@@ -1,3 +1,7 @@
# 0.1.7 (June 13, 2018)
* Move `codec::{Encode, Decode, Framed*}` into `tokio-codec` (#353)
# 0.1.6 (March 09, 2018)
* Add native endian builder fn to length_delimited (#144)
+4 -4
View File
@@ -5,10 +5,10 @@ name = "tokio-io"
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.6"
version = "0.1.7"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT/Apache-2.0"
repository = "https://github.com/tokio-rs/tokio-io"
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-io/0.1"
description = """
@@ -17,6 +17,6 @@ Core I/O primitives for asynchronous I/O in Rust.
categories = ["asynchronous"]
[dependencies]
bytes = "0.4"
bytes = "0.4.7"
futures = "0.1.18"
log = "0.4"
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2018 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+4 -11
View File
@@ -26,19 +26,12 @@ online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-io) is also a great place to get started
for the nitty-gritty.
# License
## License
This project is licensed under either of
* Apache License, Version 2.0, ([LICENSE-APACHE](../LICENSE-APACHE) or
http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](../LICENSE-MIT) or
http://opensource.org/licenses/MIT)
at your option.
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+3
View File
@@ -0,0 +1,3 @@
// For now, we need to keep the implmentation of Encoder in tokio_io.
pub use codec::Decoder;
+3
View File
@@ -0,0 +1,3 @@
// For now, we need to keep the implmentation of Encoder in tokio_io.
pub use codec::Encoder;
+262
View File
@@ -0,0 +1,262 @@
#![allow(deprecated)]
use std::io::{self, Read, Write};
use std::fmt;
use {AsyncRead, AsyncWrite};
use codec::{Decoder, Encoder};
use super::framed_read::{framed_read2, framed_read2_with_buffer, FramedRead2};
use super::framed_write::{framed_write2, framed_write2_with_buffer, FramedWrite2};
use futures::{Stream, Sink, StartSend, Poll};
use bytes::{BytesMut};
/// A unified `Stream` and `Sink` interface to an underlying I/O object, using
/// the `Encoder` and `Decoder` traits to encode and decode frames.
///
/// You can create a `Framed` instance by using the `AsyncRead::framed` adapter.
pub struct Framed<T, U> {
inner: FramedRead2<FramedWrite2<Fuse<T, U>>>,
}
pub struct Fuse<T, U>(pub T, pub U);
impl<T, U> Framed<T, U>
where T: AsyncRead + AsyncWrite,
U: Decoder + Encoder,
{
/// Provides a `Stream` and `Sink` interface for reading and writing to this
/// `Io` object, using `Decode` and `Encode` to read and write the raw data.
///
/// Raw I/O objects work with byte sequences, but higher-level code usually
/// wants to batch these into meaningful chunks, called "frames". This
/// method layers framing on top of an I/O object, by using the `Codec`
/// traits to handle encoding and decoding of messages frames. Note that
/// the incoming and outgoing frame types may be distinct.
///
/// This function returns a *single* object that is both `Stream` and
/// `Sink`; grouping this into a single object is often useful for layering
/// things like gzip or TLS, which require both read and write access to the
/// underlying object.
///
/// If you want to work more directly with the streams and sink, consider
/// calling `split` on the `Framed` returned by this method, which will
/// break them into separate objects, allowing them to interact more easily.
pub fn new(inner: T, codec: U) -> Framed<T, U> {
Framed {
inner: framed_read2(framed_write2(Fuse(inner, codec))),
}
}
}
impl<T, U> Framed<T, U> {
/// Provides a `Stream` and `Sink` interface for reading and writing to this
/// `Io` object, using `Decode` and `Encode` to read and write the raw data.
///
/// Raw I/O objects work with byte sequences, but higher-level code usually
/// wants to batch these into meaningful chunks, called "frames". This
/// method layers framing on top of an I/O object, by using the `Codec`
/// traits to handle encoding and decoding of messages frames. Note that
/// the incoming and outgoing frame types may be distinct.
///
/// This function returns a *single* object that is both `Stream` and
/// `Sink`; grouping this into a single object is often useful for layering
/// things like gzip or TLS, which require both read and write access to the
/// underlying object.
///
/// This objects takes a stream and a readbuffer and a writebuffer. These field
/// can be obtained from an existing `Framed` with the `into_parts` method.
///
/// If you want to work more directly with the streams and sink, consider
/// calling `split` on the `Framed` returned by this method, which will
/// break them into separate objects, allowing them to interact more easily.
pub fn from_parts(parts: FramedParts<T, U>) -> Framed<T, U>
{
Framed {
inner: framed_read2_with_buffer(framed_write2_with_buffer(Fuse(parts.io, parts.codec), parts.write_buf), parts.read_buf),
}
}
/// Returns a reference to the underlying I/O stream wrapped by
/// `Frame`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_ref(&self) -> &T {
&self.inner.get_ref().get_ref().0
}
/// Returns a mutable reference to the underlying I/O stream wrapped by
/// `Frame`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner.get_mut().get_mut().0
}
/// Consumes the `Frame`, returning its underlying I/O stream.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_inner(self) -> T {
self.inner.into_inner().into_inner().0
}
/// Consumes the `Frame`, returning its underlying I/O stream, the buffer
/// with unprocessed data, and the codec.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_parts(self) -> FramedParts<T, U> {
let (inner, read_buf) = self.inner.into_parts();
let (inner, write_buf) = inner.into_parts();
FramedParts {
io: inner.0,
codec: inner.1,
read_buf: read_buf,
write_buf: write_buf,
_priv: (),
}
}
}
impl<T, U> Stream for Framed<T, U>
where T: AsyncRead,
U: Decoder,
{
type Item = U::Item;
type Error = U::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
self.inner.poll()
}
}
impl<T, U> Sink for Framed<T, U>
where T: AsyncWrite,
U: Encoder,
U::Error: From<io::Error>,
{
type SinkItem = U::Item;
type SinkError = U::Error;
fn start_send(&mut self,
item: Self::SinkItem)
-> StartSend<Self::SinkItem, Self::SinkError>
{
self.inner.get_mut().start_send(item)
}
fn poll_complete(&mut self) -> Poll<(), Self::SinkError> {
self.inner.get_mut().poll_complete()
}
fn close(&mut self) -> Poll<(), Self::SinkError> {
self.inner.get_mut().close()
}
}
impl<T, U> fmt::Debug for Framed<T, U>
where T: fmt::Debug,
U: fmt::Debug,
{
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("Framed")
.field("io", &self.inner.get_ref().get_ref().0)
.field("codec", &self.inner.get_ref().get_ref().1)
.finish()
}
}
// ===== impl Fuse =====
impl<T: Read, U> Read for Fuse<T, U> {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
self.0.read(dst)
}
}
impl<T: AsyncRead, U> AsyncRead for Fuse<T, U> {
unsafe fn prepare_uninitialized_buffer(&self, buf: &mut [u8]) -> bool {
self.0.prepare_uninitialized_buffer(buf)
}
}
impl<T: Write, U> Write for Fuse<T, U> {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
self.0.write(src)
}
fn flush(&mut self) -> io::Result<()> {
self.0.flush()
}
}
impl<T: AsyncWrite, U> AsyncWrite for Fuse<T, U> {
fn shutdown(&mut self) -> Poll<(), io::Error> {
self.0.shutdown()
}
}
impl<T, U: Decoder> Decoder for Fuse<T, U> {
type Item = U::Item;
type Error = U::Error;
fn decode(&mut self, buffer: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
self.1.decode(buffer)
}
fn decode_eof(&mut self, buffer: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
self.1.decode_eof(buffer)
}
}
impl<T, U: Encoder> Encoder for Fuse<T, U> {
type Item = U::Item;
type Error = U::Error;
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
self.1.encode(item, dst)
}
}
/// `FramedParts` contains an export of the data of a Framed transport.
/// It can be used to construct a new `Framed` with a different codec.
/// It contains all current buffers and the inner transport.
#[derive(Debug)]
pub struct FramedParts<T, U> {
/// The inner transport used to read bytes to and write bytes to
pub io: T,
/// The codec
pub codec: U,
/// The buffer with read but unprocessed data.
pub read_buf: BytesMut,
/// A buffer with unprocessed data which are not written yet.
pub write_buf: BytesMut,
/// This private field allows us to add additional fields in the future in a
/// backwards compatible way.
_priv: (),
}
impl<T, U> FramedParts<T, U> {
/// Create a new, default, `FramedParts`
pub fn new(io: T, codec: U) -> FramedParts<T, U> {
FramedParts {
io,
codec,
read_buf: BytesMut::new(),
write_buf: BytesMut::new(),
_priv: (),
}
}
}
+214
View File
@@ -0,0 +1,214 @@
#![allow(deprecated)]
use std::fmt;
use AsyncRead;
use codec::Decoder;
use super::framed::Fuse;
use futures::{Async, Poll, Stream, Sink, StartSend};
use bytes::BytesMut;
/// A `Stream` of messages decoded from an `AsyncRead`.
pub struct FramedRead<T, D> {
inner: FramedRead2<Fuse<T, D>>,
}
pub struct FramedRead2<T> {
inner: T,
eof: bool,
is_readable: bool,
buffer: BytesMut,
}
const INITIAL_CAPACITY: usize = 8 * 1024;
// ===== impl FramedRead =====
impl<T, D> FramedRead<T, D>
where T: AsyncRead,
D: Decoder,
{
/// Creates a new `FramedRead` with the given `decoder`.
pub fn new(inner: T, decoder: D) -> FramedRead<T, D> {
FramedRead {
inner: framed_read2(Fuse(inner, decoder)),
}
}
}
impl<T, D> FramedRead<T, D> {
/// Returns a reference to the underlying I/O stream wrapped by
/// `FramedRead`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_ref(&self) -> &T {
&self.inner.inner.0
}
/// Returns a mutable reference to the underlying I/O stream wrapped by
/// `FramedRead`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner.inner.0
}
/// Consumes the `FramedRead`, returning its underlying I/O stream.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_inner(self) -> T {
self.inner.inner.0
}
/// Returns a reference to the underlying decoder.
pub fn decoder(&self) -> &D {
&self.inner.inner.1
}
/// Returns a mutable reference to the underlying decoder.
pub fn decoder_mut(&mut self) -> &mut D {
&mut self.inner.inner.1
}
}
impl<T, D> Stream for FramedRead<T, D>
where T: AsyncRead,
D: Decoder,
{
type Item = D::Item;
type Error = D::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
self.inner.poll()
}
}
impl<T, D> Sink for FramedRead<T, D>
where T: Sink,
{
type SinkItem = T::SinkItem;
type SinkError = T::SinkError;
fn start_send(&mut self,
item: Self::SinkItem)
-> StartSend<Self::SinkItem, Self::SinkError>
{
self.inner.inner.0.start_send(item)
}
fn poll_complete(&mut self) -> Poll<(), Self::SinkError> {
self.inner.inner.0.poll_complete()
}
fn close(&mut self) -> Poll<(), Self::SinkError> {
self.inner.inner.0.close()
}
}
impl<T, D> fmt::Debug for FramedRead<T, D>
where T: fmt::Debug,
D: fmt::Debug,
{
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("FramedRead")
.field("inner", &self.inner.inner.0)
.field("decoder", &self.inner.inner.1)
.field("eof", &self.inner.eof)
.field("is_readable", &self.inner.is_readable)
.field("buffer", &self.inner.buffer)
.finish()
}
}
// ===== impl FramedRead2 =====
pub fn framed_read2<T>(inner: T) -> FramedRead2<T> {
FramedRead2 {
inner: inner,
eof: false,
is_readable: false,
buffer: BytesMut::with_capacity(INITIAL_CAPACITY),
}
}
pub fn framed_read2_with_buffer<T>(inner: T, mut buf: BytesMut) -> FramedRead2<T> {
if buf.capacity() < INITIAL_CAPACITY {
let bytes_to_reserve = INITIAL_CAPACITY - buf.capacity();
buf.reserve(bytes_to_reserve);
}
FramedRead2 {
inner: inner,
eof: false,
is_readable: buf.len() > 0,
buffer: buf,
}
}
impl<T> FramedRead2<T> {
pub fn get_ref(&self) -> &T {
&self.inner
}
pub fn into_inner(self) -> T {
self.inner
}
pub fn into_parts(self) -> (T, BytesMut) {
(self.inner, self.buffer)
}
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
}
impl<T> Stream for FramedRead2<T>
where T: AsyncRead + Decoder,
{
type Item = T::Item;
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
loop {
// Repeatedly call `decode` or `decode_eof` as long as it is
// "readable". Readable is defined as not having returned `None`. If
// the upstream has returned EOF, and the decoder is no longer
// readable, it can be assumed that the decoder will never become
// readable again, at which point the stream is terminated.
if self.is_readable {
if self.eof {
let frame = try!(self.inner.decode_eof(&mut self.buffer));
return Ok(Async::Ready(frame));
}
trace!("attempting to decode a frame");
if let Some(frame) = try!(self.inner.decode(&mut self.buffer)) {
trace!("frame decoded from buffer");
return Ok(Async::Ready(Some(frame)));
}
self.is_readable = false;
}
assert!(!self.eof);
// Otherwise, try to read more data and try again. Make sure we've
// got room for at least one byte to read to ensure that we don't
// get a spurious 0 that looks like EOF
self.buffer.reserve(1);
if 0 == try_ready!(self.inner.read_buf(&mut self.buffer)) {
self.eof = true;
}
self.is_readable = true;
}
}
}
+237
View File
@@ -0,0 +1,237 @@
#![allow(deprecated)]
use std::io::{self, Read};
use std::fmt;
use {AsyncRead, AsyncWrite};
use codec::{Decoder, Encoder};
use super::framed::Fuse;
use futures::{Async, AsyncSink, Poll, Stream, Sink, StartSend};
use bytes::BytesMut;
/// A `Sink` of frames encoded to an `AsyncWrite`.
pub struct FramedWrite<T, E> {
inner: FramedWrite2<Fuse<T, E>>,
}
pub struct FramedWrite2<T> {
inner: T,
buffer: BytesMut,
}
const INITIAL_CAPACITY: usize = 8 * 1024;
const BACKPRESSURE_BOUNDARY: usize = INITIAL_CAPACITY;
impl<T, E> FramedWrite<T, E>
where T: AsyncWrite,
E: Encoder,
{
/// Creates a new `FramedWrite` with the given `encoder`.
pub fn new(inner: T, encoder: E) -> FramedWrite<T, E> {
FramedWrite {
inner: framed_write2(Fuse(inner, encoder)),
}
}
}
impl<T, E> FramedWrite<T, E> {
/// Returns a reference to the underlying I/O stream wrapped by
/// `FramedWrite`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_ref(&self) -> &T {
&self.inner.inner.0
}
/// Returns a mutable reference to the underlying I/O stream wrapped by
/// `FramedWrite`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner.inner.0
}
/// Consumes the `FramedWrite`, returning its underlying I/O stream.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_inner(self) -> T {
self.inner.inner.0
}
/// Returns a reference to the underlying decoder.
pub fn encoder(&self) -> &E {
&self.inner.inner.1
}
/// Returns a mutable reference to the underlying decoder.
pub fn encoder_mut(&mut self) -> &mut E {
&mut self.inner.inner.1
}
}
impl<T, E> Sink for FramedWrite<T, E>
where T: AsyncWrite,
E: Encoder,
{
type SinkItem = E::Item;
type SinkError = E::Error;
fn start_send(&mut self, item: E::Item) -> StartSend<E::Item, E::Error> {
self.inner.start_send(item)
}
fn poll_complete(&mut self) -> Poll<(), Self::SinkError> {
self.inner.poll_complete()
}
fn close(&mut self) -> Poll<(), Self::SinkError> {
Ok(try!(self.inner.close()))
}
}
impl<T, D> Stream for FramedWrite<T, D>
where T: Stream,
{
type Item = T::Item;
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
self.inner.inner.0.poll()
}
}
impl<T, U> fmt::Debug for FramedWrite<T, U>
where T: fmt::Debug,
U: fmt::Debug,
{
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("FramedWrite")
.field("inner", &self.inner.get_ref().0)
.field("encoder", &self.inner.get_ref().1)
.field("buffer", &self.inner.buffer)
.finish()
}
}
// ===== impl FramedWrite2 =====
pub fn framed_write2<T>(inner: T) -> FramedWrite2<T> {
FramedWrite2 {
inner: inner,
buffer: BytesMut::with_capacity(INITIAL_CAPACITY),
}
}
pub fn framed_write2_with_buffer<T>(inner: T, mut buf: BytesMut) -> FramedWrite2<T> {
if buf.capacity() < INITIAL_CAPACITY {
let bytes_to_reserve = INITIAL_CAPACITY - buf.capacity();
buf.reserve(bytes_to_reserve);
}
FramedWrite2 {
inner: inner,
buffer: buf,
}
}
impl<T> FramedWrite2<T> {
pub fn get_ref(&self) -> &T {
&self.inner
}
pub fn into_inner(self) -> T {
self.inner
}
pub fn into_parts(self) -> (T, BytesMut) {
(self.inner, self.buffer)
}
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
}
impl<T> Sink for FramedWrite2<T>
where T: AsyncWrite + Encoder,
{
type SinkItem = T::Item;
type SinkError = T::Error;
fn start_send(&mut self, item: T::Item) -> StartSend<T::Item, T::Error> {
// If the buffer is already over 8KiB, then attempt to flush it. If after flushing it's
// *still* over 8KiB, then apply backpressure (reject the send).
if self.buffer.len() >= BACKPRESSURE_BOUNDARY {
try!(self.poll_complete());
if self.buffer.len() >= BACKPRESSURE_BOUNDARY {
return Ok(AsyncSink::NotReady(item));
}
}
try!(self.inner.encode(item, &mut self.buffer));
Ok(AsyncSink::Ready)
}
fn poll_complete(&mut self) -> Poll<(), Self::SinkError> {
trace!("flushing framed transport");
while !self.buffer.is_empty() {
trace!("writing; remaining={}", self.buffer.len());
let n = try_ready!(self.inner.poll_write(&self.buffer));
if n == 0 {
return Err(io::Error::new(io::ErrorKind::WriteZero, "failed to
write frame to transport").into());
}
// TODO: Add a way to `bytes` to do this w/o returning the drained
// data.
let _ = self.buffer.split_to(n);
}
// Try flushing the underlying IO
try_ready!(self.inner.poll_flush());
trace!("framed transport flushed");
return Ok(Async::Ready(()));
}
fn close(&mut self) -> Poll<(), Self::SinkError> {
try_ready!(self.poll_complete());
Ok(try!(self.inner.shutdown()))
}
}
impl<T: Decoder> Decoder for FramedWrite2<T> {
type Item = T::Item;
type Error = T::Error;
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<T::Item>, T::Error> {
self.inner.decode(src)
}
fn decode_eof(&mut self, src: &mut BytesMut) -> Result<Option<T::Item>, T::Error> {
self.inner.decode_eof(src)
}
}
impl<T: Read> Read for FramedWrite2<T> {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
self.inner.read(dst)
}
}
impl<T: AsyncRead> AsyncRead for FramedWrite2<T> {
unsafe fn prepare_uninitialized_buffer(&self, buf: &mut [u8]) -> bool {
self.inner.prepare_uninitialized_buffer(buf)
}
}
+36
View File
@@ -0,0 +1,36 @@
//! Utilities for encoding and decoding frames.
//!
//! Contains adapters to go from streams of bytes, [`AsyncRead`] and
//! [`AsyncWrite`], to framed streams implementing [`Sink`] and [`Stream`].
//! Framed streams are also known as [transports].
//!
//! [`AsyncRead`]: #
//! [`AsyncWrite`]: #
//! [`Sink`]: #
//! [`Stream`]: #
//! [transports]: #
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(hidden, html_root_url = "https://docs.rs/tokio-codec/0.1.0")]
// _tokio_codec are the items that belong in the `tokio_codec` crate. However, because we need to
// maintain backward compatibility until the next major breaking change, they are defined here.
// When the next breaking change comes, they should be moved to the `tokio_codec` crate and become
// independent.
//
// The primary reason we can't move these to `tokio-codec` now is because, again for backward
// compatibility reasons, we need to keep `Decoder` and `Encoder` in tokio_io::codec. And `Decoder`
// and `Encoder` needs to reference `Framed`. So they all still need to still be in the same
// module.
mod decoder;
mod encoder;
mod framed;
mod framed_read;
mod framed_write;
pub use self::decoder::Decoder;
pub use self::encoder::Encoder;
pub use self::framed::{Framed, FramedParts};
pub use self::framed_read::FramedRead;
pub use self::framed_write::FramedWrite;
+1 -1
View File
@@ -76,6 +76,6 @@ impl<T> io::Read for AllowStdIo<T> where T: io::Read {
}
impl<T> AsyncRead for AllowStdIo<T> where T: io::Read {
// TODO: override prepare_unitialized_buffer once `Read::initializer` is stable.
// TODO: override prepare_uninitialized_buffer once `Read::initializer` is stable.
// See rust-lang/rust #42788
}

Some files were not shown because too many files have changed in this diff Show More