Compare commits

..
Author SHA1 Message Date
Carl Lerche 2d78cfe56a chore: prepare v0.2.5 release (#1984)
Also includes:
- `tokio-macros` v0.2.1
2019-12-18 13:07:27 -08:00
Artem Vorotnikov 4c645866ef stream: add next and map utility fn (#1962)
Introduces `StreamExt` trait. This trait will be used to add utility functions
to make working with streams easier. This patch includes two functions:

* `next`: a future returning the item in the stream.
* `map`: transform each item in the stream.
2019-12-18 11:57:22 -08:00
Carl Lerche b0836ece7a sync: encapsulate TryLockError variants (#1980)
As there is currently only one variant, make the error type an opaque
struct.
2019-12-18 11:50:56 -08:00
Douman 0c0f682010 Improve runtime threading options docs 2019-12-18 20:14:22 +01:00
Douman b24ad9fe86 rt: add configuration for core threads and max threads (#1977)
`num_threads` is deprecated. Instead, `core_threads` and `max_threads` are
introduced. `core_threads` specifies the number of "always on" threads used
for the async task executor and `max_threads` specifies the maximum number
of threads that the runtime may spawn.
2019-12-18 10:31:49 -08:00
Carl Lerche 7c010ed030 sync: add broadcast channel (#1943)
Adds a broadcast channel implementation. A broadcast channel is a
multi-producer, multi-consumer channel where each consumer receives a
clone of every value sent. This is useful for implementing pub / sub
style patterns.

Implemented as a ring buffer, a Vec of the specified capacity is
allocated on initialization of the channel. Values are pushed into
slots.

When the channel is full, a send overwrites the oldest value. Receivers
detect this and return an error on the next call to receive. This
prevents unbounded buffering and does not make the channel vulnerable to
the slowest consumer.

Closes: #1585
2019-12-18 10:13:15 -08:00
Dmitrii Goriunov 42c942de14 Fix ROADMAP link (#1981) 2019-12-18 09:54:35 -05:00
Kelly Thomas Kline 5e8f7eb03c docs: correct spelling (#1974) 2019-12-17 22:52:40 -08:00
Michael P. Jung 9211adbe01 sync: add Semaphore (#1973)
Provide an asynchronous Semaphore implementation. This is useful for
synchronizing concurrent access to a shared resource.
2019-12-17 22:32:12 -08:00
yim7 e5b99b0f7a sync: print MutexGuard inner value for debugging (#1961) 2019-12-17 22:02:31 -08:00
Carl Lerche 83cd754bc8 rt: fix blocking pool shutdown logic (#1978)
The blocking task queue was not explicitly drained as part of the
blocking pool shutdown logic. It was originally assumed that the
contents of the queue would be dropped when the blocking pool structure
is dropped. However, tasks must be explicitly shutdown, so we must drain
the queue can call `shutdown` on each task.

Fixes #1970, #1946
2019-12-17 21:24:26 -08:00
Ruben De Smet 17e424112d time: impl Stream for DelayQueue (#1975) 2019-12-17 21:01:29 -08:00
Carl Lerche 41d15ea212 rt: avoid dropping a task in calls to wake() (#1972)
Calls to tasks should not be nested. Currently, while a task is being
executed and the runtime is shutting down, a call to wake() can result
in the wake target to be dropped. This, in turn, results in the drop
handler being called.

If the user holds a ref cell borrow, a mutex guard, or any such value,
dropping the task inline can result in a deadlock.

The fix is to permit tasks to be scheduled during the shutdown process
and dropping the tasks once they are popped from the queue.

Fixes #1929, #1886
2019-12-17 20:52:09 -08:00
Kelly Thomas Kline 8add90210b docs: correct grammar (#1968) 2019-12-17 13:57:25 -08:00
Kelly Thomas Kline efb4b67a54 chore: add roadmap (#1965) 2019-12-16 09:39:49 -08:00
Vlad-Shcherbina 74d33a1b2f Fix typo in sync documentation (#1942) 2019-12-14 10:17:36 -08:00
Jake Goulding 69885e214c Enable the full feature when compiled for the playground (#1960)
Closes #1932
2019-12-14 10:16:20 -08:00
Artem Vorotnikov 4b85565bd7 time: stream throttle (#1949) 2019-12-13 22:06:41 -08:00
Artem Vorotnikov d593c5b051 chore: remove benches and fix/work around clippy lints (#1952) 2019-12-13 22:01:47 -08:00
Douman 91ecb4b4c2 macros: inherit visibility 2019-12-13 19:33:44 +01:00
Sean McArthur 8abaf89e5f Re-enable writev support in TcpStreams (#1956) 2019-12-13 10:25:27 -08:00
Carl Lerche b560df9e66 chore: fix warning in tokio-util tests (#1955)
`bytes` added a warning when using a fn that resulted in a useless
clone.
2019-12-13 10:03:10 -08:00
Mathspy df8278acb6 Fix typo with updated docs (#1920)
I think this is a typo
2019-12-12 14:55:25 -05:00
nickelc 5862b9a2e0 chore: fix the outdated example in README (#1930) 2019-12-11 14:12:53 -08:00
Carl Lerche c0953d41a5 chore: fix thread_pool benchmarks (#1947)
Update the rotted thread_pool benchmarks. These benchmarks are not the
greatest, but as of now it is all we have for micro benchmarks.

Adds a little yielding in the parker as it helps a bit.
2019-12-11 12:44:45 -08:00
Michael HowellandTaiki Endo 24cd6d67f7 io: add AsyncSeek trait (#1924)
Co-authored-by: Taiki Endo <[email protected]>
2019-12-10 21:48:24 -08:00
Michael P. Jung 975576952f Add Mutex::try_lock and (Unbounded)Receiver::try_recv (#1939) 2019-12-10 08:01:23 -08:00
Juan Alvarez 5d5755dca4 fix spawn function documentation (#1940) 2019-12-10 10:46:48 -05:00
Danilo Bargen 41ffdbb7d9 sync::Mutex: Fix typo in documentation (#1934) 2019-12-09 15:07:21 -05:00
Danilo Bargen 2450b5bfc9 sync::Mutex: Add note about the absence of poisoning (#1933) 2019-12-09 09:13:24 -08:00
Carl Lerche 80abff0e57 chore: prepare v0.2.4 release (#1917)
Includes a `Mutex` bug fix
2019-12-06 19:50:29 -08:00
Michael P. Jung c632337e6f sync: fix Mutex when lock future dropped before complete (#1902)
The bug caused the mutex to reach a state where it is locked and cannot be unlocked.

Fixes #1898
2019-12-06 14:30:02 -08:00
Carl Lerche a53f94ab61 doc: expand on runtime / spawn docs (#1914) 2019-12-06 13:10:18 -08:00
Carl Lerche 98c9a77f18 prepare v0.2.3 release (#1912) 2019-12-06 09:47:28 -08:00
Carl Lerche e00c49611a doc: fix TcpListener example to compile (#1911)
The `process_socket` is hidden from the user which makes the example
fail to compile if copied by the reader.
2019-12-06 09:16:08 -08:00
Jeremy Kolb 9c9fabc44b Close markdown (#1910) 2019-12-06 07:51:24 -08:00
Steven Fackler c3461b3ef3 time: impl From between std / tokio Instants (#1904) 2019-12-05 12:03:04 -08:00
Eliza Weisman b7ecd35036 task: fix LocalSet failing to poll all local futures (#1905)
Currently, a `LocalSet` does not notify the `LocalFuture` again at the
end of a tick. This means that if we didn't poll every task in the run
queue during that tick (e.g. there are more than 61 tasks enqueued),
those tasks will not be polled.

This commit fixes this issue by changing `local::Scheduler::tick` to
return whether or not the local future needs to be notified again, and
waking the task if so.

Fixes #1899
Fixes #1900

Signed-off-by: Eliza Weisman <[email protected]>
2019-12-05 12:00:10 -08:00
Kevin Leimkuhler dbcd1f9a09 time: Remove HandlePriv (#1896)
## Motivation

#1800 removed the lazy binding of `Delay`s to timers. With the removal of the
logic required for that, `HandlePriv` is no longer needed. This PR removes the
use of `HandlePriv`.

A `TODO` was also removed that would panic if when registering a new `Delay`
the current timer handle was full. That has been fixed to now immediately
transition that `Delay` to an error state that can be handled in a similar way
to other error states.

Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-12-04 20:46:16 -08:00
Eliza Weisman 0e729aa341 task: fix infinite loop when dropping a LocalSet (#1892)
## Motivation

There's currently an issue in `task::LocalSet` where dropping the local
set can result in an infinite loop if a task running in the local set is
notified from outside the local set (e.g. by a timer). This was reported
in issue #1885.

This issue exists because the `Drop` impl for `task::local::Scheduler`
does not drain the queue of tasks notified externally, the way the basic
scheduler does. Instead, only the local queue is drained, leaving some
tasks in place. Since these tasks are never removed, the loop that
continues trying to cancel tasks until the owned task list is totally
empty continues infinitely.

I think this issue was due to the `Drop` impl being written before a
remote queue was added to the local scheduler, and the need to close the
remote queue as well was overlooked.

## Solution

This branch solves the problem by clearing the local scheduler's remote
queue as well as the local one.

I've added a test that reproduces the behavior. The test fails on master
and passes after this change.

In addition, this branch factors out the common task queue logic in the
basic scheduler runtime and the `LocalSet` struct in `tokio::task`. This
is because as more work was done on the `LocalSet`, it has gotten closer
and closer to the basic scheduler in behavior, and factoring out the
shared code reduces the risk of errors caused by `LocalSet` not doing
something that the basic scheduler does. The queues are now encapsulated
by a `MpscQueues` struct in `tokio::task::queue` (crate-public).  As a
follow-up, I'd also like to look into changing this type to use the same
remote queue type as the threadpool (a linked list).

In particular, I noticed the basic scheduler has a flag that indicates
the remote queue has been closed, which is set when dropping the
scheduler. This prevents tasks from being added after the scheduler has
started shutting down, stopping a potential task leak. Rather than
duplicating this code in `LocalSet`, I thought it was probably better to
factor it out into a shared type.

There are a few cases where there are small differences in behavior,
though, so there is still a need for separate types implemented _using_
the new `MpscQueues` struct. However, it should cover most of the 
identical code.

Note that this diff is rather large, due to the refactoring. However, the
actual fix for the infinite loop is very simple. It can be reviewed on its own
by looking at commit 4f46ac6. The refactor is in a separate commit, with
the SHA 90b5b1f.

Fixes #1885

Signed-off-by: Eliza Weisman <[email protected]>
2019-12-04 11:20:57 -08:00
Artem Vorotnikov cbe369a3ed Make JoinError Sync (#1888)
* Make JoinError Sync

* Move Mutex inside JoinError internals, hide its constructors

* Deprecate JoinError constructors, fix internal usages
2019-12-04 10:51:23 -08:00
Juan Alvarez 8bcbe78dbe remove io workarounds from example (#1891)
This PR removes no longer needed io workarounds from connect example.
2019-12-03 16:07:09 -08:00
Christopher Coverdale 6efe07c3fb Fixing minor spelling mistake in task docs (#1889) 2019-12-03 12:01:59 -08:00
Xinkai Chen 8a2160a913 Add unit tests for tokio::File::AsRaw{Fd,Handle} for Unix and Windows. (#1890)
Supersedes #1640.
2019-12-03 09:56:32 -08:00
baizhenxuan 38c361781f examples: fix tinyhttp (#1884) 2019-12-02 20:28:36 -08:00
Eliza Weisman 07451f8b94 task: relax 'static bound in LocalSet::block_on (#1882)
## Motivation

Currently, `tokio::task::LocalSet`'s `block_on` method requires the
future to live for the 'static lifetime. However, this bound is not
required — the future is wrapped in a `LocalFuture`, and then passed
into `Runtime::block_on`, which does _not_ require a `'static` future.

This came up while updating `tokio-compat` to work with version 0.2. To
mimic the behavior of `tokio` 0.1's `current_thread::Runtime::run`, we
want to be able to have a runtime block on the `recv` future from an
mpsc channel indicating when the runtime is idle. To support `!Send`
futures, as the old `current_thread::Runtime` did, we must do so inside
of a `LocalSet`. However, with the current bounds, we cannot await an
`mpsc::Receiver`'s `recv` future inside the `LocalSet::block_on` call.

## Solution

This branch removes the unnecessary `'static` bound.

Signed-off-by: Eliza Weisman <[email protected]>
2019-12-02 16:43:33 -08:00
Carl Lerche e87df0557d io: add async fns for reading / writing bufs (#1881)
Adds `read_buf` and `write_buf` which work with `T: BufMut` and `T: Buf`
respectively. This adds an easy API for using the buffer traits provided
by `bytes.
2019-12-02 13:09:31 -08:00
Carl Lerche a8a4a9f0fc blocking: fix spawn_blocking after shutdown (#1875)
The task handle needs to be shutdown explicitly and not dropped.

Closes #1853
2019-12-01 12:58:01 -08:00
Carl Lerche 8b60c5386a doc: fix documented feature flags for tokio::task (#1876)
Some feature flags are missing and some are duplicated.

Closes #1836
2019-12-01 12:49:38 -08:00
Carl Lerche af07f5bee7 sync: expand oneshot docs and TryRecvError (#1874)
`oneshot::Receiver::try_recv` does not provide any information as to the
reason **why** receiving failed. The two cases are that the channel is
empty or that the channel closed.

`TryRecvError` is changed to be an enum of those two cases. This is
backwards compatible as `TryRecvError` was an opaque struct.

This also expands on `oneshot` API documentation, adding details and
examples.

Closes #1872
2019-12-01 10:48:47 -08:00
Ivan Petkov 939a0dd7b0 process: rewrite and simplify the issue_42 test (#1871) 2019-11-30 15:17:04 -08:00
Carl Lerche 1ea6733568 io: read/write big-endian numbers (#1863)
Provide convenience methods for encoding and decoding big-endian numbers
on top of asynchronous I/O streams. Only primitive types are provided
(24 and 48 bit numbers are omitted).

In general, using these methods won't be the fastest way to do
encoding/decoding with asynchronous byte streams, but they help to get
simple things working fast.
2019-11-30 13:13:21 -08:00
Carl Lerche 8ce408492a doc: improve AsyncBufReadExt API documentation (#1868)
Remove "old" docs that were left over during a rewrite, add examples and
additional details.
2019-11-30 13:12:39 -08:00
Carl Lerche b559a0cd9a net: expose TcpStream::poll_peek (#1864)
This used to be exposed in 0.1, but was switched to private during the
upgrade. The `async fn` is sufficient for many, but not all cases.

Closes #1556
2019-11-30 09:36:03 -08:00
Carl Lerche 417460cf86 doc: expand mpsc::Sender::send API documentation (#1865)
Includes more description, lists errors, and examples.

Closes #1579
2019-11-30 09:35:23 -08:00
Carl Lerche adaba1a0bc doc: add API docs for AsyncBufReadExt::read_line (#1866)
Include more details and an example.

Closes #1592
2019-11-30 09:34:42 -08:00
Ivan Petkov 467b6ea783 chore: prepare v0.2.2 release (#1857) 2019-11-29 11:09:28 -08:00
Carl Lerche a2cfc877a7 rt: fix basic_scheduler notification bug (#1861)
The "global executor" thread-local is to track where to spawn new tasks,
**not** which scheduler is active on the current thread. This fixes a
bug with scheduling tasks on the basic_scheduler by tracking the
currently active basic_scheduler with a dedicated thread-local variable.

Fixes: #1851
2019-11-29 10:23:22 -08:00
Ömer Sinan Ağacan ec7f2ae306 docs: Mention features for basic_scheduler, threaded_scheduler (#1858)
Fixes #1829
2019-11-29 08:26:58 -08:00
Bartek Iwańczuk 4261ab6627 fs: add File::into_std and File::try_into_std methods (#1856)
In version 0.1 there was File::into_std method that destructured
tokio_fs::File into std::fs:File. That method was lacking in
version 0.2.

Fixes: #1852
2019-11-28 17:09:28 -08:00
Ivan Petkov aef434c089 signal: update documentation with caveats (#1854) 2019-11-28 15:03:04 -08:00
Ömer Sinan Ağacan cd73951130 Implement Stream for signal::unix::Signal (#1849)
Refs #1848
2019-11-28 08:54:50 -08:00
Eliza Weisman 524e66314f task: fix panic when dropping LocalSet (#1843)
It turns out that the `Scheduler::release` method on `LocalSet`'s
`Scheduler` *is* called, when the  `Scheduler` is dropped with tasks
still running. Currently, that method is `unreachable!`, which means
that dropping a `LocalSet` with tasks running will panic.

This commit fixes the panic, by pushing released tasks to
`pending_drop`. This is the same as `BasicScheduler`.

Fixes #1842
2019-11-27 14:24:44 -08:00
Michael Zeller 34d751bf92 net: fix ucred for illumos/solaris (#1772) 2019-11-27 12:22:22 -08:00
Oleg Nosov 942feab040 doc: misc API documentation fixes (#1834) 2019-11-27 12:05:42 -08:00
Oleg Nosov dc356a4158 doc: fix runtime::Builder example (#1841) 2019-11-27 12:03:57 -08:00
Oleg Nosov 2cd1d74092 rt: specify that runtime should have task scheduler (#1839)
* Specify that runtime should have task scheduler

* Even more detailed panic message for incorrect task spawn
2019-11-27 10:25:21 -08:00
Carl Lerche 632ee507ba prepare v0.2.1 release (#1832)
This includes `task::LocalSet` as well as some misc small fixes.
2019-11-26 21:46:02 -08:00
Carl Lerche 7f605ee27f doc: fix and improve incoming() API doc (#1831)
This fixes the API docs for both `TcpListener::incoming` and
`UnixListener::incoming`. The function now takes `&mut self` instead of
`self`. Adds an example for both function.
2019-11-26 21:15:13 -08:00
Eliza Weisman 38e602f4d8 task: add LocalSet API for running !Send futures (#1733)
## Motivation

In earlier versions of `tokio`, the `current_thread::Runtime` type could
be used to run `!Send` futures. However, PR #1716 merged the
current-thread and threadpool runtimes into a single type, which can no
longer run `!Send` futures. There is still a need in some cases to
support futures that don't implement `Send`, and the `tokio-compat`
crate requires this in order to provide APIs that existed in `tokio`
0.1.

## Solution

This branch implements the API described by @carllerche in
https://github.com/tokio-rs/tokio/pull/1716#issuecomment-549496309. It
adds a new `LocalSet` type and `spawn_local` function to `tokio::task`.
The `LocalSet` type is used to group together a set of tasks which must
run on the same thread and don't implement `Send`. These are available
when a new "rt-util" feature flag is enabled.

Currently, the local task set is run by passing it a reference to a
`Runtime` and a future to `block_on`. In the future, we may also want
to investigate allowing spawned futures to construct their own local
task sets, which would be executed on the worker that the future is
executing on. 

In order to implement the new API, I've made some internal changes to
the `task` module and `Schedule` trait to support scheduling both `Send`
and `!Send` futures.

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-26 17:03:18 -08:00
Artem Vorotnikov 8e83a9f2c3 chore: replace Gitter badge with Discord (#1828) 2019-11-26 16:00:38 -08:00
Carl Lerche c146f48f0b fs: impl AsRawFd / AsRawHandle for File (#1827)
This provides the ability to get the raw OS handle for a `File`. The
`Into*` variant cannot be provided as `File` needs to maintain ownership
of the `File`. The actual handle may have been moved to a background
thread.
2019-11-26 16:00:26 -08:00
Benjamin Fry ebf5f37989 time: reexport Elapsed (#1826) 2019-11-26 15:10:41 -08:00
Carl Lerche abfa857f09 chore: remove updating note from readme (#1824) 2019-11-26 10:36:17 -08:00
Carl Lerche a81e2722a4 chore: prepare v0.2.0 release (#1822) 2019-11-26 09:17:27 -08:00
Carl Lerche 4ddc437170 doc: add more doc_cfg annotations (#1821)
Also makes the `tokio::net::{tcp, udp, unix}` modules only for "utility"
types. The primary types are in `tokio::net` directly.
2019-11-25 14:32:55 -08:00
Carl Lerche 3ecaa6d91c docs: improve tokio::io API documentation (#1815)
Adds method level documentation for `tokio::io`.
2019-11-23 08:24:03 -08:00
leo-lb 0bc68adb34 tokio: remove performance regression notice (#1817) 2019-11-23 07:45:56 -08:00
Ivan Petkov e20dff39ce process: do not kill spawned processes on drop (#1814)
This updates the tokio `Command` and `Child` behavior to match that of
the stdlib: spawned processes will *not* be automatically killed when
the handle is dropped

Unlike the stdlib, any dropped (unix) processes may be reaped by tokio
behind-the-scenes after they exit and if new processes are awaited,
which mitigates the risks of piling up unreaped zombie unix processes

A `Command::kill_on_drop` method is added to allow the caller to
control whether the spawned child should be killed when the handle is
dropped. By default, this value is `false`.

The `Child::forget` method has been removed, as it is superseded by
`Command::kill_on_drop`
2019-11-22 20:10:05 -08:00
Carl Lerche 7b4c999341 default all feature flags to off (#1811)
Changes the set of `default` feature flags to `[]`. By default, only
core traits are included without specifying feature flags. This makes it
easier for users to pick the components they need.

For convenience, a `full` feature flag is included that includes all
components.

Tests are configured to require the `full` feature. Testing individual
feature flags will need to be moved to a separate crate.

Closes #1791
2019-11-22 15:55:10 -08:00
Carl Lerche e1b1e216c5 ci: bring back build tests (#1813)
This directory was deleted when `cargo hack` was introduced, however
there were some tests that were still useful (macro failure output).

Also, additional build tests will be added over time.
2019-11-22 14:38:49 -08:00
Taiki Endo 7cd63fb946 ci: use -Z avoid-dev-deps in features check instead of --no-dev-deps (#1812) 2019-11-22 14:13:18 -08:00
Carl Lerche bf741fec35 ci: generate docs (#1810)
Check docs as part of CI. This should catch link errors.
2019-11-22 11:55:57 -08:00
Carl Lerche 9b2aa14bb1 docs: annotate io mod with doc_cfg (#1808)
Annotates types in `tokio::io` module with their required feature flag.
This annotation is included in generated documentation.

Notes:

* The annotation must be on the type or function itself. Annotating just
  the re-export is not sufficient.

* The annotation must be **inside** the `pin_project!` macro or it is
  lost.
2019-11-22 09:56:08 -08:00
Carl Lerche 8546ff826d runtime: cleanup and add config options (#1807)
* runtime: cleanup and add config options

This patch finishes the cleanup as part of the transition to Tokio 0.2.
A number of changes were made to take advantage of having all Tokio
types in a single crate. Also, fixes using Tokio types from
`spawn_blocking`.

* Many threads, one resource driver

Previously, in the threaded scheduler, a resource driver (mio::Poll /
timer combo) was created per thread. This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from **outside** of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.

Now, instead of creating many resource drivers, a single resource driver
is used. Each scheduler thread will attempt to "lock" the resource
driver before parking on it. If the resource driver is already locked,
the thread uses a condition variable to park. Contention should remain
low as, under load, the scheduler avoids using the drivers.

* Add configuration options to enable I/O / time

New configuration options are added to `runtime::Builder` to allow
enabling I/O and time drivers on a runtime instance basis. This is
useful when wanting to create lightweight runtime instances to execute
compute only tasks.

* Bug fixes

The condition variable parker is updated to the same algorithm used in
`std`. This is motivated by some potential deadlock cases discovered by
`loom`.

The basic scheduler is fixed to fairly schedule tasks. `push_front` was
accidentally used instead of `push_back`.

I/O, time, and spawning now work from within `spawn_blocking` closures.

* Misc cleanup

The threaded scheduler is no longer generic over `P :Park`. Instead, it
is hard coded to a specific parker. Tests, including loom tests, are
updated to use `Runtime` directly. This provides greater coverage.

The `blocking` module is moved back into `runtime` as all usage is
within `runtime` itself.
2019-11-21 23:28:39 -08:00
Eliza Weisman 6866fe426c docs: expand and update crate-level docs (#1806)
## Motivation

Tokio's crate-level docs are currently pretty sparse, and in some cases
reference old names for APIs. Before 0.2 is released, they could use a
fresh coat of paint.

## Solution

This branch reworks and expands the `lib.rs` docs. In particular, I've
added a new "A Tour of Tokio" section, inspired by the [standard
library's similarly-named section][std]. This section lists all of
`tokio`'s public modules, and summarizes their major APIs. It also lists
the feature flags necessary to enable those APIs.

[std]: https://doc.rust-lang.org/std/index.html#a-tour-of-the-rust-standard-library

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-21 14:09:10 -08:00
Eliza Weisman d88846c4eb docs: update and expand the tokio::runtime API docs (#1804)
## Motivation

The `tokio::runtime` module's docs need to be updated to
track recent changes.

## Solution

This branch updates and expands the `runtime` docs.

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-20 17:46:35 -08:00
Eliza Weisman 7e6a10fccd docs: refresh tokio::io API docs (#1803)
## Motivation

The `tokio::io` module's docs are fairly sparse and not particularly up
to date. They ought to be improved before release.

## Solution

This branch adds new module-level docs to `tokio::io`. The new docs are
largely inspired by `std::io`'s documentation, and highlight the
similarities and differences between `tokio::io` and `std::io`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-20 15:09:38 -08:00
Carl Lerche 502cf5d95c io: flatten split module (#1802) 2019-11-20 14:45:38 -08:00
Eliza Weisman c223db3589 docs: improve tokio::task API documentation (#1801)
## Motivation

The new `tokio::task` module is pretty lacking in API docs. 

## Solution

This branch adds new API docs to the `task` module, including:

* Module-level docs with a summary of the differences between 
  tasks and threads
* Examples of how to use the `task` APIs in the module-level docs
* More docs for `yield_now`
* More docs and examples for `JoinHandle`, based on the 
  `std::thread::JoinHandle` API docs.

This branch contains commits cherry-picked from #1794 

Signed-off-by: Eliza Weisman <[email protected]>
2019-11-20 14:36:45 -08:00
Carl Lerche 5cd665afd7 chore: update bytes dependency to git master (#1796)
Tokio will track changes to bytes until 0.5 is released.
2019-11-20 14:27:49 -08:00
Kevin Leimkuhler 3e643c7b81 time: Eagerly bind delays to timer (#1800)
## Motivation

Similar to #1666, it is no longer necessary to lazily register delays with the
executions default timer. All delays are expected to be created from within a
runtime, and should panic if not done so.

## Solution

`tokio::time` now assumes there to be a `CURRENT_TIMER` set when creating a
delay; this can be assumed if called within a tokio runtime. If there is no
current timer, the application will panic with a "no current timer" message.

## Follow-up

Similar to #1666, `HandlePriv` can probably be removed, but this mainly prepares
for 0.2 API changes. Because it is not in the public API, this can be done in a
following change.

Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-11-20 12:24:41 -08:00
Pen Tree bc150cd0b5 Fix doc links (#1799)
Link fix only. After this fix, `cargo doc --package` succeeds.
2019-11-20 12:24:17 -08:00
Carl Lerche 15dce2d11a net: flatten split mod (#1797)
The misc `split` types (`ReadHalf`, `WriteHalf`, `SendHalf`, `RecvHalf`)
are moved up a module and the `*::split` module is removed.
2019-11-20 11:29:32 -08:00
Taiki Endo d4fec2c5d6 chore: enable feature flag check on windows (#1798) 2019-11-20 07:05:50 -08:00
Carl Lerche 69975fb960 Refactor the I/O driver, extracting slab to tokio::util. (#1792)
The I/O driver is made private and moved to `tokio::io::driver`. `Registration` is
moved to `tokio::io::Registration` and `PollEvented` is moved to `tokio::io::PollEvented`.

Additionally, the concurrent slab used by the I/O driver is cleaned up and extracted to
`tokio::util::slab`, allowing it to eventually be used by other types.
2019-11-20 00:05:14 -08:00
Carl Lerche 7c8b8877d4 runtime: fix lost wakeup bug in scheduler (#1788)
When checking if a worker needs to be unparked, the SeqCst load does not
provide the necessary synchronization to ensure the scheduled task is
visible to the searching worker. The `load` is switched to
`fetch_add(0)` which does establish the necessary synchronization.

Adding unit tests catching this bug will require a fix to loom and will
be done at a later time. The bug fix has been validated with manual
testing.

Fixes #1768
2019-11-19 08:01:46 -08:00
Carl Lerche 0d38936b35 chore: refine feature flags (#1785)
Removes dependencies between Tokio feature flags. For example, `process`
should not depend on `sync` simply because it uses the `mpsc` channel.
Instead, feature flags represent **public** APIs that become available
with the feature enabled. When the feature is not enabled, the
functionality is removed. If another Tokio component requires the
functionality, it is stays as `pub(crate)`.

The threaded scheduler is now exposed under `rt-threaded`. This feature
flag only enables the threaded scheduler and does not include I/O,
networking, or time. Those features must be explictly enabled.

A `full` feature flag is added that enables all features.

`stdin`, `stdout`, `stderr` are exposed under `io-std`.

Macros are used to scope code by feature flag.
2019-11-18 07:00:55 -08:00
sclaire-1 13b6e9939e Edit CONTRIBUTING.md (#1784)
Edited the last sentence of the first section to improve clarity
2019-11-17 23:27:42 -08:00
Carl Lerche 44f10fe47f sync: require T: Clone for watch channels. (#1783)
There are limitations with `async/await` (no GAT) requiring the value to
be cloned on receive. The `poll` based API is not currently exposed.
This makes the `Clone` requirement explicit.
2019-11-17 09:03:44 -08:00
Carl Lerche c147be0437 make AtomicWaker private (#1782) 2019-11-16 23:35:17 -08:00
Carl Lerche b1d9e55487 task: move blocking fns into tokio::task (#1781) 2019-11-16 23:35:04 -08:00
Taiki Endo 66cbed3ce3 tls: enable test on CI (#1779) 2019-11-16 22:24:58 -08:00
Carl Lerche 4d19a99937 runtime: set spawn context on enter (#1780) 2019-11-16 22:24:28 -08:00
Taiki Endo 10dc659450 io: expose std{in, out, err} under io feature (#1759)
This exposes `std{in, out, err}` under io feature by moving
`fs::blocking` module into `io::blocking`.
As `fs` feature depends on `io-trait` feature, `fs` implementations can
always access `io` module.
2019-11-16 22:03:39 -08:00
Taiki Endo 320c84a433 chore: migrate from pin-project to pin-project-lite (#1778) 2019-11-16 09:14:40 -08:00
Carl Lerche 19f1fc36bd task: return JoinHandle from spawn (#1777)
`tokio::spawn` now returns a `JoinHandle` to obtain the result of the task:

Closes #887.
2019-11-16 08:28:34 -08:00
Carl Lerche 3f0eabe779 runtime: rename current_thread -> basic_scheduler (#1769)
It no longer supports executing !Send futures. The use case for
It is wanting a “light” runtime. There will be “local” task execution
using a different strategy coming later.

This patch also renames `thread_pool` -> `threaded_scheduler`, but
only in public APIs for now.
2019-11-16 07:19:45 -08:00
Taiki Endo 1474794055 runtime: allow non-unit type output in {Runtime, Spawner}::spawn (#1756) 2019-11-15 22:16:21 -08:00
Taiki Endo 92eb635669 net: add more impls for ToSocketAddrs (#1760) 2019-11-15 22:12:57 -08:00
Carl Lerche 8a7e57786a Limit futures dependency to Stream via feature flag (#1774)
In an effort to reach API stability, the `tokio` crate is shedding its
_public_ dependencies on crates that are either a) do not provide a
stable (1.0+) release with longevity guarantees or b) match the `tokio`
release cadence. Of course, implementing `std` traits fits the
requirements.

The on exception, for now, is the `Stream` trait found in `futures_core`.
It is expected that this trait will not change much and be moved into `std.
Since Tokio is not yet going reaching 1.0, I feel that it is acceptable to maintain
a dependency on this trait given how foundational it is.

Since the `Stream` implementation is optional, types that are logically
streams provide `async fn next_*` functions to obtain the next value.
Avoiding the `next()` name prevents fn conflicts with `StreamExt::next()`.

Additionally, some misc cleanup is also done:

- `tokio::io::io` -> `tokio::io::util`.
- `delay` -> `delay_until`.
- `Timeout::new` -> `timeout(...)`.
- `signal::ctrl_c()` returns a future instead of a stream.
- `{tcp,unix}::Incoming` is removed (due to lack of `Stream` trait).
- `time::Throttle` is removed (due to lack of `Stream` trait).
-  Fix: `mpsc::UnboundedSender::send(&self)` (no more conflict with `Sink` fns).
2019-11-15 22:11:13 -08:00
Markus Westerlind 930679587a codec: Remove Unpin requirement from Framed[Read,Write,] (#1758)
cc #1252
2019-11-15 16:30:07 +09:00
Carl Lerche 27e5b41067 reorganize modules (#1766)
This patch started as an effort to make `time::Timer` private. However, in an
effort to get the build compiling again, more and more changes were made. This
probably should have been broken up, but here we are. I will attempt to
summarize the changes here.

* Feature flags are reorganized to make clearer. `net-driver` becomes
  `io-driver`. `rt-current-thread` becomes `rt-core`.

* The `Runtime` can be created without any executor. This replaces `enter`. It
  also allows creating I/O / time drivers that are standalone.

* `tokio::timer` is renamed to `tokio::time`. This brings it in line with `std`.

* `tokio::timer::Timer` is renamed to `Driver` and made private.

* The `clock` module is removed. Instead, an `Instant` type is provided. This
  type defaults to calling `std::time::Instant`. A `test-util` feature flag can
  be used to enable hooking into time.

* The `blocking` module is moved to the top level and is cleaned up.

* The `task` module is moved to the top level.

* The thread-pool's in-place blocking implementation is cleaned up.

* `runtime::Spawner` is renamed to `runtime::Handle` and can be used to "enter"
  a runtime context.
2019-11-12 15:23:40 -08:00
Anton Barkovsky e3df2eafd3 tls: fix test certificate to work on macOS 10.15 (#1763)
macOS 10.15 introduced new requirements for certificates to be trusted:
https://support.apple.com/en-us/HT210176
2019-11-11 12:09:14 +01:00
Taiki Endo c15e01a09b chore: remove rust-toolchain and add minimum supported version check (#1748)
* remove rust-toolchain

* add minimum supported version check
2019-11-08 13:26:08 +09:00
Taiki Endo 64f2bf0072 chore: update CI config to test on stable (#1747) 2019-11-08 00:32:04 +09:00
Carl Lerche 7e35922a1d time: rename tokio::timer -> tokio::time (#1745) 2019-11-06 23:53:46 -08:00
Carl Lerche 4dbe6af0a1 runtime: misc pool cleanup (#1743)
- Remove builders for internal types
- Avoid duplicating the blocking pool when using the concurrent
  scheduler.
- misc smaller cleanup
2019-11-06 21:29:10 -08:00
leo-lb 9bec094150 timer: have example use delay_for instead of delay (#1735)
It is a more common use case that is to simply cause a delay for an amount of time.
I think it is more appropriate to show off `delay_for` in the example rather than `delay` that is useful only for less common use cases.
2019-11-06 21:28:21 -08:00
Taiki Endo 6f8b986bdb chore: update futures to 0.3.0 (#1741) 2019-11-07 05:09:10 +09:00
Carl Lerche 1a7f6fb201 simplify enter (#1736) 2019-11-06 09:51:15 -08:00
Carl Lerche 0da23aad77 fix clippy (#1737) 2019-11-05 23:38:52 -08:00
Carl Lerche d5c1119c88 runtime: combine executor and runtime mods (#1734)
Now, all types are under `runtime`. `executor::util` is moved to a top
level `util` module.
2019-11-05 19:12:30 -08:00
Carl Lerche a6253ed05a chore: unify all mocked loom files (#1732)
When the crates were merged, each component kept its own `loom` file
containing mocked types it needed. This patch unifies them all in one
location.
2019-11-04 22:22:40 -08:00
Carl Lerche 94f9b04b06 executor: switch some APIs to crate private. (#1731)
* switch `enter` to crate private
* make executor types pub(crate)
2019-11-04 14:12:24 -08:00
Carl Lerche 966ccd5d53 test: unify MockTask and task::spawn (#1728)
Delete `MockTask` in favor of `task::spawn`. Both are functionally
equivalent.
2019-11-03 14:10:14 -08:00
Taiki Endo 3948e16292 ci: install minimal profile by default (#1729) 2019-11-03 12:08:07 -08:00
Sebastian Dröge 6b35a1e8b0 impl AsyncWrite for std::io::Cursor (#1730)
Based on the implementation from the futures crate.
2019-11-03 21:21:01 +09:00
Carl Lerche e19bd77ef0 tests: fix bug + reorganize tests. (#1726)
Fixes a bug in the thread-pool executor related to shutdown
concurrent with a task that is self-notifying. A `loom` test is
added to validate the fix.

Additionally, in anticipation of the `thread_pool` module being
switched to private, tests are updated to use `Runtime` directly
instead of `thread_pool`. Those tests that cannot be updated
are switched to unit tests.
2019-11-02 17:03:06 -07:00
Carl Lerche c8fdbed27a chore: prune dev-dependencies
Most dev dependendencies are unused now that examples are moved into a
separate crate.
2019-11-02 09:40:37 +01:00
Carl Lerche 3e7d0be51d executor: remove Executor & TypedExecutor traits (#1724)
The `Executor` trait is sub-optimal as it forces a `Box<dyn Future>` to
spawn. Instead, `tokio::spawn` delegates to the specific runtime
implementation set for the current execution context.

`TypedExecutor`, while useful, has seen limited adoption. As such, it is
removed from `tokio` proper. Moving it to `tokio-util` is a possibility
that can be explored as follow up work.
2019-11-01 13:50:17 -07:00
Carl Lerche d70c928d88 runtime: merge multi & single threaded runtimes (#1716)
Simplify Tokio's runtime construct by combining both Runtime variants
into a single type. The execution style can be controlled by a
configuration setting on `Builder`.

The implication of this change is that there is no longer any way to
spawn `!Send` futures. This, however, is a temporary limitation. A
different strategy will be employed for supporting `!Send` futures.

Included in this patch is a rework of `task::JoinHandle` to support
using this type from both the thread-pool and current-thread executors.
2019-11-01 13:18:52 -07:00
Steven Fackler 742d89b0f3 Fix delay construction from non-lazy Handles (#1720)
Closes #1719.
2019-11-01 12:32:57 -07:00
Carl Lerche 20993341bd compat: extract crate to a dedicated git repo (#1723)
The compat crate is moved to https://github.com/tokio-rs/tokio-compat.
This allows pinning it to specific revisions of the Tokio git
repository. The master branch is intended to go through significant
churn and it will be easier to update the compat layer in batches.
2019-11-01 12:30:12 -07:00
Eliza Weisman e699d46534 compat: add a compat runtime (#1663)
## Motivation

The `futures` crate's [`compat` module][futures-compat] provides
interoperability between `futures` 0.1 and `std::future` _future types_
(e.g. implementing `std::future::Future` for a type that implements the
`futures` 0.1 `Future` trait). However, this on its own is insufficient
to run code written against `tokio` 0.1 on a `tokio` 0.2 runtime, if
that code also relies on `tokio`'s runtime services. If legacy tasks are
executed that rely on `tokio::timer`, perform IO using `tokio`'s
reactor, or call `tokio::spawn`, those API calls will fail unless there
is also a runtime compatibility layer.

## Solution

As proposed in #1549, this branch introduces a new `tokio-compat` crate,
with implementations of the thread pool and current-thread runtimes that
are capable of running both tokio 0.1 and tokio 0.2 tasks. The compat
runtime creates a background thread that runs a `tokio` 0.1 timer and
reactor, and sets itself as the `tokio` 0.1 executor as well as the
default 0.2 executor. This allows 0.1 futures that use 0.1 timer,
reactor, and executor APIs may run alongside `std::future` tasks on the
0.2 runtime.

### Examples

Spawning both `tokio` 0.1 and `tokio` 0.2 futures:

```rust
use futures_01::future::lazy;

tokio_compat::run(lazy(|| {
    // spawn a `futures` 0.1 future using the `spawn` function from the
    // `tokio` 0.1 crate:
    tokio_01::spawn(lazy(|| {
        println!("hello from tokio 0.1!");
        Ok(())
    }));

    // spawn an `async` block future on the same runtime using `tokio`
    // 0.2's `spawn`:
    tokio_02::spawn(async {
        println!("hello from tokio 0.2!");
    });

    Ok(())
}))
```

Futures on the compat runtime can use `timer` APIs from both 0.1 and 0.2
versions of `tokio`:

```rust
use std::time::{Duration, Instant};
use futures_01::future::lazy;
use tokio_compat::prelude::*;

tokio_compat::run_03(async {
    // Wait for a `tokio` 0.1 `Delay`...
    let when = Instant::now() + Duration::from_millis(10);
    tokio_01::timer::Delay::new(when)
        // convert the delay future into a `std::future` that we can `await`.
        .compat()
        .await
        .expect("tokio 0.1 timer should work!");
    println!("10 ms have elapsed");

    // Wait for a `tokio` 0.2 `Delay`...
    let when = Instant::now() + Duration::from_millis(20);
    tokio_02::timer::delay(when).await;
    println!("20 ms have elapsed");
});
```

## Future Work

This is just an initial implementation of a `tokio-compat` crate; there
are more compatibility layers we'll want to provide before that crate is
complete. For example, we should also provide compatibility between
`tokio` 0.2's `AsyncRead` and `AsyncWrite` traits and the `futures` 0.1
and `futures` 0.3 versions of those traits. In #1549, @carllerche also
suggests that the `compat` crate provide reimplementations of APIs that
were removed from `tokio` 0.2 proper, such as the `tcp::Incoming`
future.

Additionally, there is likely extra work required to get the 
`tokio-threadpool` 0.1 `blocking` APIs to work on the compat runtime.
This will be addressed in a follow-up PR.

Fixes: #1605
Fixes: #1552
Refs: #1549

[futures-compat]: https://rust-lang-nursery.github.io/futures-api-docs/0.3.0-alpha.19/futures/compat/index.html
2019-11-01 10:35:02 -07:00
Carl Lerche 72caede7be chore: remove dead files (#1718)
The `codec` module has been moved to `tokio-util`. Some files were left,
but they were never activated.
2019-11-01 21:30:06 +09:00
Carl Lerche 64c26ab1ee runtime: test creating a single-threaded runtime. (#1717) 2019-10-31 22:28:31 -07:00
Taiki Endo 02f7264008 chore: check each feature works properly (#1695)
It is hard to maintain features list manually, so use cargo-hack's
`--each-feature` flag. And cargo-hack provides a workaround for an issue
that dev-dependencies leaking into normal build (`--no-dev-deps` flag),
so removed own ci tool.

Also, compared to running tests on all features, there is not much
advantage in running tests on each feature, so only the default features
and all features are tested.
If the behavior changes depending on the feature, we need to test it as
another job in CI.
2019-10-31 21:09:32 -07:00
Jonathan Bastien-Filiatrault 2902e39db0 Allow non-destructive access to the read buffer. (#1600)
I need this to implement SMTP pipelining checks. I mostly need to
flush my send buffer when the read buffer is empty before waiting for
the next command.
2019-10-31 10:36:24 -04:00
Steven Fackler 630d3136dd timere: make Delay must_use (#1714)
Closes #1711
2019-10-30 20:21:03 -07:00
Sean McArthur 2c870b588f process: refactor OrphanQueue to use a Mutex instead fo SegQueue (#1712) 2019-10-30 15:29:04 -07:00
Jon Gjengset 109fd3086b thread-pool: in-place blocking with new scheduler (#1681)
The initial new scheduler PR omitted in-place blocking
support. This patch brings it back.
2019-10-30 08:58:49 -07:00
Sean McArthur e3261440e5 timer: inline CachePadded type (#1706) 2019-10-29 22:16:11 -07:00
Carl Lerche 2b909d6805 sync: move into tokio crate (#1705)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The sync implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-29 15:11:31 -07:00
Carl Lerche c62ef2d232 executor: move into tokio crate (#1702)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The executor implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-28 21:40:29 -07:00
Eliza Weisman 7eb264a0d0 net: replace RwLock<Slab> with a lock free slab (#1625)
## Motivation

The `tokio_net::driver` module currently stores the state associated
with scheduled IO resources in a `Slab` implementation from the `slab`
crate. Because inserting items into and removing items from `slab::Slab`
requires mutable access, the slab must be placed within a `RwLock`. This
has the potential to be a performance bottleneck especially in the context of
the work-stealing scheduler where tasks and the reactor are often located on
the same thread.

`tokio-net` currently reimplements the `ShardedRwLock` type from
`crossbeam` on top of `parking_lot`'s `RwLock` in an attempt to squeeze
as much performance as possible out of the read-write lock around the
slab. This introduces several dependencies that are not used elsewhere.

## Solution

This branch replaces the `RwLock<Slab>` with a lock-free sharded slab
implementation. 

The sharded slab is based on the concept of _free list sharding_
described by Leijen, Zorn, and de Moura in [_Mimalloc: Free List
Sharding in Action_][mimalloc], which describes the implementation of a
concurrent memory allocator. In this approach, the slab is sharded so
that each thread has its own thread-local list of slab _pages_. Objects
are always inserted into the local slab of the thread where the
insertion is performed. Therefore, the insert operation needs not be
synchronized.

However, since objects can be _removed_ from the slab by threads other
than the one on which they were inserted, removal operations can still
occur concurrently. Therefore, Leijen et al. introduce a concept of
_local_ and _global_ free lists. When an object is removed on the same
thread it was originally inserted on, it is placed on the local free
list; if it is removed on another thread, it goes on the global free
list for the heap of the thread from which it originated. To find a free
slot to insert into, the local free list is used first; if it is empty,
the entire global free list is popped onto the local free list. Since
the local free list is only ever accessed by the thread it belongs to,
it does not require synchronization at all, and because the global free
list is popped from infrequently, the cost of synchronization has a
reduced impact. A majority of insertions can occur without any
synchronization at all; and removals only require synchronization when
an object has left its parent thread.

The sharded slab was initially implemented in a separate crate (soon to
be released), vendored in-tree to decrease `tokio-net`'s dependencies.
Some code from the original implementation was removed or simplified,
since it is only necessary to support `tokio-net`'s use case, rather
than to provide a fully generic implementation.

[mimalloc]: https://www.microsoft.com/en-us/research/uploads/prod/2019/06/mimalloc-tr-v1.pdf

## Performance

These graphs were produced by out-of-tree `criterion` benchmarks of the
sharded slab implementation.


The first shows the results of a benchmark where an increasing number of
items are inserted and then removed into a slab concurrently by five
threads. It compares the performance of the sharded slab implementation
with a `RwLock<slab::Slab>`:

<img width="1124" alt="Screen Shot 2019-10-01 at 5 09 49 PM" src="https://user-images.githubusercontent.com/2796466/66078398-cd6c9f80-e516-11e9-9923-0ed6292e8498.png">

The second graph shows the results of a benchmark where an increasing
number of items are inserted and then removed by a _single_ thread. It
compares the performance of the sharded slab implementation with an
`RwLock<slab::Slab>` and a `mut slab::Slab`.

<img width="925" alt="Screen Shot 2019-10-01 at 5 13 45 PM" src="https://user-images.githubusercontent.com/2796466/66078469-f0974f00-e516-11e9-95b5-f65f0aa7e494.png">

Note that while the `mut slab::Slab` (i.e. no read-write lock) is
(unsurprisingly) faster than the sharded slab in the single-threaded
benchmark, the sharded slab outperforms the un-contended
`RwLock<slab::Slab>`. This case, where the lock is uncontended and only
accessed from a single thread, represents the best case for the current
use of `slab` in `tokio-net`, since the lock cannot be conditionally
removed in the single-threaded case.

These benchmarks demonstrate that, while the sharded approach introduces
a small constant-factor overhead, it offers significantly better
performance across concurrent accesses.

## Notes

This branch removes the following dependencies `tokio-net`:
- `parking_lot`
- `num_cpus`
- `crossbeam_util`
- `slab`

This branch adds the following dev-dependencies:
- `proptest`
- `loom`

Note that these dev dependencies were used to implement tests for the
sharded-slab crate out-of-tree, and were necessary in order to vendor
the existing tests. Alternatively, since the implementation is tested
externally, we _could_ remove these tests in order to avoid picking up
dev-dependencies. However, this means that we should try to ensure that
`tokio-net`'s vendored implementation doesn't diverge significantly from
upstream's, since it would be missing a majority of its tests.

Signed-off-by: Eliza Weisman <[email protected]>
2019-10-28 11:30:45 -07:00
Geoff Shannon 1195263584 Fix docs links: Redux (#1698) 2019-10-27 09:37:07 -07:00
Carl Lerche bccb713d98 thread-pool: test additional shutdown cases (#1697)
This adds an extra spawned task during the thread-pool shutdown loom
test. This results in additional cases being tested, primarily tasks
being stolen.
2019-10-26 22:15:39 -07:00
Linus Färnstrand 474befd23c chore: use argument position impl trait (#1690) 2019-10-26 08:40:38 -07:00
Carl Lerche 987ba7373c io: move into tokio crate (#1691)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `io` implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-26 08:02:49 -07:00
Carl Lerche 227533d456 net: move into tokio crate (#1683)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `net` implementation is now provided by the main `tokio` crate.
Functionality can be opted out of by using the various net related
feature flags.
2019-10-25 12:50:15 -07:00
Jon Gjengset 03a9378297 Make blocking pool non-static and use for thread pool (#1678)
Previously, support for `blocking` was done through a static `POOL` that
would spawn threads on demand. While this made the pool accessible at
all times, it made it hard to configure, and it was impossible to keep
multiple blocking pools.

This patch changes `blocking` to instead use a "default" global like the
ones used for timers, executors, and the like. There is now
`blocking::with_pool`, which is used by both thread-pool workers and the
current-thread runtime to ensure that a pool is available to tasks.

This patch also changes `ThreadPool` to spawn its worker threads on the
blocking pool rather than as free-standing threads. This is in
preparation for the coming in-place blocking work.

One downside of this change is that thread names are no longer
"semantic". All threads are named by the pool name, and individual
threads are not (currently) given names with numerical suffixes like
before.
2019-10-24 14:17:47 -07:00
Carl Lerche 99940aeeb4 chore: remove tracing. (#1680)
Historically, logging has been added haphazardly. Here, we entirely
remove logging as none of it is particularly useful. In the future, we
will add tracing back in order to expose useful data to the user of
Tokio.
2019-10-23 11:04:14 -07:00
Carl Lerche cfc15617a5 codec: move into tokio-util (#1675)
Related to #1318, Tokio APIs that are "less stable" are moved into a new
`tokio-util` crate. This crate will mirror `tokio` and provide
additional APIs that may require a greater rate of breaking changes.

As examples require `tokio-util`, they are moved into a separate
crate (`examples`). This has the added advantage of being able to avoid
example only dependencies in the `tokio` crate.
2019-10-22 10:13:49 -07:00
Carl Lerche b8cee1a60a timer: move tokio-timer into tokio crate (#1674)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `timer` implementation is now provided by the main `tokio` crate.
The `timer` functionality may still be excluded from the build by
skipping the `timer` feature flag.
2019-10-21 16:45:13 -07:00
Kevin Leimkuhler c9bcbe77b9 net: Eagerly bind resources to reactors (#1666)
## Motivation

The `tokio_net` resources can be created outside of a runtime due to how tokio
has been used with futures to date. For example, this allows a `TcpStream` to be
created, and later passed into a runtime:

```
let stream = TcpStream::connect(...).and_then(|socket| {
    // do something
});
tokio::run(stream);
```

In order to support this functionality, the reactor was lazily bound to the
resource on the first call to `poll_read_ready`/`poll_write_ready`. This
required a lot of additional complexity in the binding logic to support.

With the tokio 0.2 common case, this is no longer necessary and can be removed.
All resources are expected to be created from within a runtime, and should panic
if not done so.

Closes #1168

## Solution

The `tokio_net` crate now assumes there to be a `CURRENT_REACTOR` set on the
worker thread creating a resource; this can be assumed if called within a tokio
runtime. If there is no current reactor, the application will panic with a "no
current reactor" message.

With this assumption, all the unsafe and atomics have been removed from
`tokio_net::driver::Registration` as it is no longer needed.

There is no longer any reason to pass in handles to the family of `from_std` methods on `net` resources. `Handle::current` has therefore a more restricted private use where it is only used in `driver::Registration::new`.

Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-10-21 16:20:06 -07:00
Carl Lerche 978013a215 fs: move into tokio (#1672)
A step towards collapsing Tokio sub crates into a single `tokio`
crate (#1318).

The `fs` implementation is now provided by the main `tokio` crate. The
`fs` functionality may still be excluded from the build by skipping the
`fs` feature flag.
2019-10-21 15:49:00 -07:00
madmaxio 6aa6ebb5bc io: Take struct re-export to main crate (#1670) 2019-10-21 10:03:05 -07:00
Jonathas Conceição 4bee94eb06 runtime: update doc regarding runtime::run function helper (#1671) 2019-10-21 10:02:36 -07:00
Carl Lerche ed5a94eb2d executor: rewrite the work-stealing thread pool (#1657)
This patch is a ground up rewrite of the existing work-stealing thread
pool. The goal is to reduce overhead while simplifying code when
possible.

At a high level, the following architectural changes were made:

- The local run queues were switched for bounded circle buffer queues.
- Reduce cross-thread synchronization.
- Refactor task constructs to use a single allocation and always include
  a join handle (#887).
- Simplify logic around putting workers to sleep and waking them up.

**Local run queues**

Move away from crossbeam's implementation of the Chase-Lev deque. This
implementation included unnecessary overhead as it supported
capabilities that are not needed for the work-stealing thread pool.
Instead, a fixed size circle buffer is used for the local queue. When
the local queue is full, half of the tasks contained in it are moved to
the global run queue.

**Reduce cross-thread synchronization**

This is done via many small improvements. Primarily, an upper bound is
placed on the number of concurrent stealers. Limiting the number of
stealers results in lower contention. Secondly, the rate at which
workers are notified and woken up is throttled. This also reduces
contention by preventing many threads from racing to steal work.

**Refactor task structure**

Now that Tokio is able to target a rust version that supports
`std::alloc` as well as `std::task`, the pool is able to optimize how
the task structure is laid out. Now, a single allocation per task is
required and a join handle is always provided enabling the spawner to
retrieve the result of the task (#887).

**Simplifying logic**

When possible, complexity is reduced in the implementation. This is done
by using locks and other simpler constructs in cold paths. The set of
sleeping workers is now represented as a `Mutex<VecDeque<usize>>`.
Instead of optimizing access to this structure, we reduce the amount the
pool must access this structure.

Secondly, we have (temporarily) removed `threadpool::blocking`. This
capability will come back later, but the original implementation was way
more complicated than necessary.

**Results**

The thread pool benchmarks have improved significantly:

Old thread pool:

```
test chained_spawn ... bench:   2,019,796 ns/iter (+/- 302,168)
test ping_pong     ... bench:   1,279,948 ns/iter (+/- 154,365)
test spawn_many    ... bench:  10,283,608 ns/iter (+/- 1,284,275)
test yield_many    ... bench:  21,450,748 ns/iter (+/- 1,201,337)
```

New thread pool:

```
test chained_spawn ... bench:     147,943 ns/iter (+/- 6,673)
test ping_pong     ... bench:     537,744 ns/iter (+/- 20,928)
test spawn_many    ... bench:   7,454,898 ns/iter (+/- 283,449)
test yield_many    ... bench:  16,771,113 ns/iter (+/- 733,424)
```

Real-world benchmarks improve significantly as well. This is testing the hyper hello
world server using: `wrk -t1 -c50 -d10`:

Old scheduler:

```
Running 10s test @ http://127.0.0.1:3000
  1 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   371.53us   99.05us   1.97ms   60.53%
    Req/Sec   114.61k     8.45k  133.85k    67.00%
  1139307 requests in 10.00s, 95.61MB read
Requests/sec: 113923.19
Transfer/sec:      9.56MB
```

New scheduler:

```
Running 10s test @ http://127.0.0.1:3000
  1 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   275.05us   69.81us   1.09ms   73.57%
    Req/Sec   153.17k    10.68k  171.51k    71.00%
  1522671 requests in 10.00s, 127.79MB read
Requests/sec: 152258.70
Transfer/sec:     12.78MB
```
2019-10-19 11:09:40 -07:00
Steven Fackler 2a181320b7 fs: add read_to_string (#1664) 2019-10-16 15:47:37 -07:00
Taiki Endo 4c97e9dc28 fs: remove unnecessary trait and lifetime bounds (#1655) 2019-10-15 19:02:34 +09:00
Jon Gjengset 1cae04f8b3 macros: Use more consistent runtime names (#1628)
As discussed in #1620, the attribute names for `#[tokio::main]` and
`#[tokio::test]` aren't great. Specifically, they both use
`single_thread` and `multi_thread`, as opposed to names that match the
runtime names: `current_thread` and `threadpool`. This PR changes the
former to the latter.

Fixes #1627.
2019-10-12 12:55:39 -04:00
John-John Tedro 29f35df7f8 Remove incorrect FusedFuture impl on Delay (#1652)
`is_terminated` must return `true` until the future has been polled at least once to make sure that the associated block in select is called even after the delay has elapsed.

You use `Delay` in a `select!` by [fusing it](https://docs.rs/futures-preview/0.3.0-alpha.19/futures/future/trait.FutureExt.html#method.fuse):

```rust
let delay = tokio::timer::delay(/* ... */);
let delay = delay.fuse();

select! {
    _ = delay => {
        /* work here */
    }
}
```
2019-10-11 15:45:44 -04:00
Ivan Petkov 741bef8fe1 tokio: move signal and process reexports to crate root (#1643) 2019-10-11 11:00:39 -07:00
Carl Lerche 804dbd6f8e sync: fix mem leak in oneshot on task migration (#1648)
When polling the task, the current waker is saved to the oneshot state.
When the handle is migrated to a new task and polled again, the waker
must be swaped from the old waker to the new waker. In some cases, there
is a potential for the old waker to leak.

This bug was caught by loom with the recently added memory leak
detection.
2019-10-10 12:00:22 -07:00
Eliza Weisman 69fe65e972 io: add AsyncBufReadExt::split (#1642)
add a `split` method to `AsyncBufReadExt`, analogous to `std::io::BufRead::split`.
2019-10-09 13:17:07 -07:00
Jonathan Bastien-Filiatrault b8913ec7c0 executor: accurate idle thread tracking for the blocking pool (#1621)
Use a counter to count notifications. This protects against spurious
wakeups by pthreads and other libraries. The state transitions now
track num_idle precisely.
2019-10-07 14:04:28 -07:00
Eliza Weisman 8aa520e2bd io: add missing utility functions (#1632)
The standard library's `io` module has small utilities such as `repeat`,
`empty`, and `sink`, which return `Read` and `Write` implementations.
These can come in handy in some circiumstances. `tokio::io` has no
equivalents that implement `AsyncRead`/`AsyncWrite`.

This commit adds `repeat`, `empty`, and `sink` helpers to `tokio::io`.
2019-10-07 14:02:04 -07:00
Nick Stott ab2f71a612 chore: fix a comment typo (#1633) 2019-10-07 09:20:57 -07:00
Taiki Endo 42a5cb1508 timer: test arm on targets with target_has_atomic less than 64 (#1634) 2019-10-07 09:19:44 -07:00
Taiki Endo 2b4b0619d7 chore: update Cirrus CI config to test on beta (#1636) 2019-10-07 09:18:38 -07:00
Taiki Endo 55caddb9ce chore: do not trigger CI on std-future branch (#1635) 2019-10-07 09:17:27 -07:00
Vojtech Kral aefaef3abf tcp: export Incoming type (#1602) 2019-10-02 11:12:05 -07:00
Jon Gjengset c78c9168d7 macros: allow selecting runtime in tokio::test attr (#1620)
In the past, it was not possible to choose to use the multi-threaded
tokio `Runtime` in tests, which meant that any test that transitively
used `executor::threadpool::blocking` would fail with

```
'blocking' annotation used from outside the context of a thread pool
```

This patch adds a runtime annotation attribute to `#[tokio::test]` just
like `#[tokio::main]` has, which lets users opt in to the threadpool
runtime over `current_thread` (the default).
2019-10-02 10:58:34 -07:00
Jonathan Bastien-Filiatrault 9e1eef829a chore: annotate prelude re-exports as doc(no_inline) (#1601)
Fixes #1593 by making "use as _" linked in the documentation.
2019-10-02 10:55:35 -07:00
Taiki Endo f48980ae52 chore: update rust-toolchain to use beta (#1619) 2019-10-01 10:13:38 -04:00
Douman a1d1eb5eb3 macros: Allow arguments in non-main functions 2019-10-01 13:15:46 +02:00
Jon Gjengset 5efe31f2ed Prepare for release of 0.2.0-alpha.6 (#1617)
Note that `tokio-timer` and `tokio-tls` become 0.3.0-alpha.6 (not 0.2.0)
2019-09-30 18:35:52 -04:00
Jon Gjengset 5ce5a0a0e0 Fix for rust-lang/rust#64477 (#1618)
`foo(format!(...)).await` no longer compiles. There's a fix in
rust-lang/rust#64856, but this works around the problem.
2019-09-30 17:17:14 -04:00
Jon Gjengset 5fd5329497 Create BufStream from a BufReader + BufWriter (#1609)
This is handy if developers want to construct the inner buffers with a
particular capacity, and still end up with a `BufStream` at the end.
2019-09-30 14:22:59 -04:00
Taiki Endo 3b8ee2d991 chore: update futures-preview to 0.3.0-alpha.19 (#1610) 2019-09-30 13:32:37 -04:00
Jon Gjengset 7c341f45e0 chore: move CI to beta (#1615) 2019-09-27 09:51:45 -07:00
Jon Gjengset 611b4e11a7 Make Barrier::wait future Send (#1611)
It wasn't before. Now it is. And that is better.
2019-09-26 18:26:24 -04:00
Taiki Endo 159abb375f chore: update pin-project to 0.4 (#1603) 2019-09-27 04:51:28 +09:00
Carl Lerche 032b39487c sync: add spin_loop_hint to atomic waker (#1608)
The algorithm backing `AtomicWaker` effectively uses a spin lock backed
by notifying & yielding the current task. This adds a `spin_lock_hint`
annotation to cover this case.

While, in practice, the omission of `spin_lock_hint` would not cause
problems, there are platforms that do not handle spin locks very well
and could enter a deadlock in pathological cases.
2019-09-26 15:16:34 -04:00
Hung-I Wang b71b7b36be fs: update the doc comment of File::sync_data (#1596) 2019-09-25 09:05:50 -07:00
Taiki Endo c4567f741a io: add get_*/into_inner methods to BufStream (#1598) 2019-09-25 09:17:43 -04:00
Sean McArthur 18cef1901f tokio: add rt-current-thread optional feature
- Adds a minimum `rt-current-thread` optional feature that exports
  `tokio::runtime::current_thread`.
- Adds a `macros` optional feature to enable the `#[tokio::main]` and
  `#[tokio::test]` attributes.
- Adjusts `#[tokio::main]` macro to select a runtime "automatically" if
  a specific strategy isn't specified. Allows using the macro with only
  the rt-current-thread feature.
2019-09-24 12:17:04 -07:00
Taiki Endo c81447fdcc io: remove unsafe pin-projections and remove manual Unpin implementations (#1588)
* Removes most pin-projection related unsafe code.

* Removes manual Unpin implementations.
  As references always implement Unpin, there is no need to implement
  Unpin manually.

* Adds tests to check that Unpin requirement does not change accidentally 
  because changing Unpin requirements will be breaking changes.
2019-09-25 01:17:06 +09:00
Taiki Endo d50d050fae net: fix build-tests for uds (#1589) 2019-09-24 02:47:39 +09:00
Taiki Endo 3a55aba251 macros: add build tests for #[tokio::main] and #[tokio::test] (#1591) 2019-09-23 04:09:30 +09:00
Taiki Endo ddbb0c3836 macros: fix handling of arguments of #[tokio::main] attribute (#1578) 2019-09-23 03:05:04 +09:00
Taiki Endo 376d63867a chore: update pin-project to 0.4.0-beta.1 (#1586) 2019-09-23 01:52:14 +09:00
Taiki Endo eb2d0fbcd1 net: use Box::pin instead of Pin::new(Box::new) (#1587) 2019-09-22 09:28:55 -07:00
Jonathan Bastien-Filiatrault 695165feac timer: 32 bit ARM only has 32 bit atomics. (#1581) 2019-09-20 13:25:49 -07:00
Kirill Mironov ff186a4d03 tokio: add process feature (#1561) 2019-09-19 19:03:58 -07:00
Jon Gjengset 6611b32cce Export sync::Barrier from tokio::sync (#1577) 2019-09-19 21:32:35 -04:00
Carl Lerche 80ba2a4ff6 Release 0.2.0 alpha.5 (#1576) 2019-09-19 13:39:35 -07:00
Carl Lerche 8d09f61d33 net: fix build with only process (#1575) 2019-09-19 12:38:15 -07:00
Carl Lerche 815173f8e5 chore: rm tokio-buf (#1574)
The crate has not been updated and it does not seem like it is a good
path forward.
2019-09-19 12:11:21 -07:00
Markus Westerlind 34e388619f timer: delay_for should use tokio_timer::clock::now (#1572) 2019-09-19 11:20:18 -07:00
Jon Gjengset 9d5af20bcf Enable buffering both reads and writes (#1558)
`BufWriter` and `BufReader` did not previously forward the "opposite" trait (`AsyncRead` for `BufWriter` and `AsyncWrite` for `BufReader`). This meant that there was no way to have both directions buffered at once. This patch fixes that, and introduces a convenience type + constructor for this double-wrapped construct.
2019-09-19 14:17:15 -04:00
Jon Gjengset 613fde2637 sync: add Barrier primitive (#1571)
This adds `Barrier` to `tokio-sync`, which is an asynchronous alternative to [`std::sync::Barrier`](https://doc.rust-lang.org/std/sync/struct.Barrier.html). It is a synchronization primitive that allows multiple futures to "rendezvous" at certain points in their execution.
2019-09-19 14:16:56 -04:00
Jonathan Bastien-Filiatrault 22a3b10171 executor: fix blocking pool bug re: thread shutdown (#1562)
Currently, when threads in the blocking pool shutdown due to being idle
the counter tracking threads is not decremented. This prevents new threads
from being spawned to replace the shutdown threads.
2019-09-19 11:12:04 -07:00
Jon Gjengset e3415d8d61 sync: Make Lock more similar to std::sync::Mutex (#1573)
This renames `Lock` to `Mutex`, and brings the API more in line with `std::sync::Mutex`.

In partcular, locking now only takes `&self`, with the expectation that you place the `Mutex` in an `Arc` (or something similar) to share it between threads.

Fixes #1544.
Part of #1210.
2019-09-19 11:46:52 -04:00
Taiki Endo d1f60ac4c6 chore: deny warnings for doc tests (#1539) 2019-09-19 15:50:12 +09:00
Taiki Endo e2161502ad chore: fix clippy check failure (#1569) 2019-09-18 10:19:44 -07:00
yjh ab785bfba7 Update README.md (#1545)
change url's `version` to `latest`.
2019-09-17 10:54:22 -04:00
Lucio Franco 5f2f3f076d Add broken feature to old benchmarks (#1555)
Signed-off-by: Lucio Franco <[email protected]>
2019-09-13 14:04:23 -04:00
Taiki Endo efb27731ad timer: use our own AtomicU64 on targets with target_has_atomic less than 64 (#1538) 2019-09-13 10:18:32 -07:00
cynecx 578a9aec16 sync: replace deprecated mem::uninitialized usage with MaybeUninit (#1540) 2019-09-13 10:03:03 -07:00
Jonathan Bastien-Filiatrault 5b8fc19701 fs: propagate flush for stdout / stderr. (#1528) 2019-09-13 09:58:18 -07:00
Geoff Shannon c0a64d67ca chore: fix docs links (#1523) 2019-09-13 09:46:19 -07:00
Carl Lerche 6369d0f4f2 chore: add stability note to readme. (#1554) 2019-09-13 09:12:30 -07:00
Kirill Mironov f69ee652e6 tls: fix new temporary lifetime rustc error [E0597] (#1547)
Fixes: #1546
Signed-off-by: Kirill Mironov <[email protected]>
2019-09-11 10:22:04 -07:00
Jon Gjengset 9b3f8564af tls: Add get_ref and get_mut (#1537) 2019-09-04 17:45:53 -04:00
Ivan Petkov 9766cd644f process: omit several future types in favor of async/await (#1526) 2019-08-31 13:02:27 -07:00
Carl Lerche 431d4857e8 io: add Send / Sync impls for ReadHalf / WriteHalf (#1525) 2019-08-31 12:18:55 -07:00
Fenhl 26432355d5 tokio-process: Implement From<StdCommand> for Command (#1513) 2019-08-31 11:45:41 -07:00
Carl Lerche 2f91c85ad8 io: bring back split utility (#1521)
Bring back `split` utility as a free fn instead of a method on
`AsyncRead`. This utility wraps the `stream` in an `Arc` and uses mutual
exclusion to ensure correct access.

Additionally, the specialized `split_mut` fn on TcpStream and UdsStream
is promoted to `split`.
2019-08-30 20:46:07 -07:00
Fenhl 951827229a Add platform-specific methods to Command (#1516) 2019-08-30 18:28:41 -07:00
Geoff Shannon 383bb0a143 test: fix assert format args (#1520) 2019-08-30 14:03:37 -07:00
Benjamin Saunders d2bd6f5002 timer: Rename sleep to delay_for, reexport from tokio (#1518) 2019-08-30 10:23:54 -07:00
Carl Lerche 6a94d2cf4f tls: bump to v0.3.0-alpha.4 (#1515) 2019-08-30 10:20:44 -07:00
kellerkindt 4f99470d46 chore: fix compile error on latest nightly (#1512) 2019-08-30 09:15:05 -07:00
Jarred Nicholls 3d9134d13e executor: shut down idle threads in the blocking pool (#1514) 2019-08-30 08:26:16 -07:00
Sean McArthur 15dc0563b7 prepare v0.2.0-alpha.4 (#1509) 2019-08-29 12:59:10 -07:00
Sean McArthur 4e26258ac3 Re-add temporarily TcpStream::connect_std (#1508) 2019-08-29 11:45:17 -07:00
Carl Lerche a59e096c47 prepare v0.2.0-alpha.3 release (#1505) 2019-08-28 15:04:42 -07:00
Carl Lerche fc1640891e net: perform DNS lookup on connect / bind. (#1499)
A sealed `net::ToSocketAddrs` trait is added. This trait is not intended
to be used by users. Instead, it is an argument to `connect` and `bind`
functions.

The operating system's DNS lookup functionality is used. Blocking
operations are performed on a thread pool in order to avoid blocking the
runtime.
2019-08-28 13:25:50 -07:00
Jakub Beránek de9f05d4d3 docs: fix wording in tokio_process::Child documentation (#1502)
Fixes: #1494
2019-08-28 11:56:42 -04:00
Eliza Weisman 9c31797a08 net: switch from log to tracing (#1455)
* net: switch from `log` to `tracing`.

Motivation:

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime.

Solution:

This branch replaces the use of `log` in `tokio-net` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation.

Notes:

I removed the timing in `Reactor::poll` in favor of simply adding a
`#[tracing::instrument]` attribute. Since the generated `tracing` span
will have enter and exit events, a `tracing::Subscriber`
implemementation can use those to record timestamps, and process that
timing data in a much more sophisticated manner than including it in a
log line.

We can add the timestamps back if they're desired.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-27 17:53:57 -07:00
Ömer Sinan Ağacan d1c58b7940 tokio: export RunError in tokio::runtime::current_thread (#1487)
This type is used in return type of `Runtime::run`, but because the type
was not exported it was opaque in the documentation of `Runtime`.
2019-08-27 12:26:48 -07:00
Jacob Pratt 5f74a99ea3 implement spawn_with_handle in tokio_executor (#1492)
This code directly relies on `future-preview`'s `RemoteHandle`, and
exposes it via a `spawn_with_handle` method that is identical to
`future-preview`'s implementation.

Related: #1180
2019-08-27 12:26:11 -07:00
Carl Lerche 08e20fcf6a fs: add support for non-threadpool executors (#1495)
Provides a thread pool dedicated to running blocking operations (#588)
and update `tokio-fs` to use this pool.

In an effort to make incremental progress, this is an initial step
towards a final solution. First, it provides a very basic pool
implementation with the intend that the pool will be
replaced before the final release. Second, it updates `tokio-fs` to
always use this blocking pool instead of conditionally using
`threadpool::blocking`. Issue #588 contains additional discussion around
potential improvements to the "blocking for all" strategy.

The implementation provided here builds on work started in #954 and
continued in #1045. The general idea is th same as #1045, but the PR
improves on some of the details:

* The number of explicit operations tracked by `File` is reduced only to
  the ones that could interact. All other ops are spawned on the
  blocking pool without being tracked by the `File` instance.

* The `seek` implementation is not backed by a trait and `poll_seek`
  function. This avoids the question of how to model non-blocking seeks
  on top of a blocking file. In this patch, `seek` is represented as an
  `async fn`. If the associated future is dropped before the caller
  observes the return value, we make no effort to define the state in
  which the file ends up.
2019-08-27 12:25:20 -07:00
Carl Lerche 08099bb2d3 net: rewrite TcpStream::connect with async fn (#1497)
This also removes `TcpStream::connect_std` as the conversion functions
from `std` need to be rethought. A note tracking this has been added
to #1209.
2019-08-27 12:08:28 -07:00
Newton Ni 807d536846 codec: fix infinite loop in tokio_codec::LinesCodec (#1489) 2019-08-26 13:38:52 -07:00
Danny Browning 654f9d703f tokio: expose signal feature (#1491)
Expose tokio_net::signal::ctrl_c via tokio::net::signal::ctrl_c as feature signal.
2019-08-22 09:12:00 -07:00
Jon Gjengset a285689664 net: shutdown TCP write when asked to shut down (#1488) 2019-08-21 10:41:51 -07:00
Gurwinder Singh 13930eff2a chore: two async_await feature remained (#1486) 2019-08-21 18:53:42 +09:00
Taiki Endo 24fb33e012 io: add AsyncReadExt::{chain, take} (#1484) 2019-08-20 20:09:07 -07:00
Taiki Endo a791f4a758 chore: bump to newer nightly (#1485) 2019-08-20 20:07:16 -07:00
Eliza Weisman 7e7a5147a3 executor: switch from log to tracing (#1454)
## Motivation

The `tracing` crate implements scoped, structured, context-aware
diagnostics, which can add significant debugging value over unstructured
log messages. `tracing` is part of the Tokio project. As part of the
`tokio` 0.2 changes, I thought it would be good to move over from `log`
to `tracing` in the tokio runtime. Updating the executor crate is an obvious
starting point. 

## Solution

This branch replaces the use of `log` in `tokio-executor` with
`tracing`. I've tried to leave all the instrumentation points more or
less the same, but modified to use structured fields instead of string
interpolation. I've also added a few `tracing` spans, primarily in
places where a variable is added to all the log messages in a scope.

## Notes

For users who are using the legacy `log` output, there is a feature flag
to enable `log` support in `tracing`. I thought about making this on by
default, but that would also enable the `tracing` dependency by default,
and it is only pulled in when the `threadpool` feature flag is enabled.
The `tokio` crate could enable the log feature in its default features
instead, since the threadpool feature is on by default in `tokio`. If
this isn't the right approach, I can change how `log` back-compatibility
is enabled.

We might want to consider adding more `tracing` spans in the threadpool
later. This could be useful for profiling, and for helping users debug
the way their applications interact with the executor. This branch is
just intended as a starting point so that we can begin emitting
`tracing` data from the executor; we should revisit what instrumentation
should be exposed, as well.

Signed-off-by: Eliza Weisman <[email protected]>
2019-08-20 12:44:26 -07:00
Jakub Beránek 2d56312b89 timer: introduce delay function shortcut (#1440)
This commit adds a simple delay shortcut to avoid writing Delay::new
everywhere and removes usages of Delay::new.
2019-08-20 08:39:55 -07:00
Ivan Petkov 357df38861 process: move into the tokio-net crate (#1475) 2019-08-19 19:42:54 -07:00
John-John Tedro 34a9dc2d76 Implement FusedStream and FusedFuture for Interval and Delay (#1476) 2019-08-19 10:21:34 -04:00
Ivan Petkov 68d5fcb8d1 docs: fix all rustdoc warnings (#1474) 2019-08-18 14:38:54 -07:00
Ivan Petkov 08b07afbd9 signal: remove new() constructors in favor of free functions (#1472)
* Also removed any `*_with_handle` related methods in favor of always
using the default reactor
2019-08-18 14:22:09 -07:00
Douman 7b0c60849c net: make default reactor guard public (#1468) 2019-08-18 11:36:21 -07:00
Ivan Petkov 6d8d388dc5 docs: add docs.rs metadata to build with all features (#1471) 2019-08-18 11:11:46 -07:00
Ivan Petkov bc61bd9d3d ci: ensure all tests are run for each feature (#1470)
* This includes running docs, examples, and lib tests for each added
feature, to ensure nothing is broken
2019-08-18 10:50:38 -07:00
Jakub Beránek a9585f0318 tokio-process: change CommandExt to a fully asynchronous Command struct (#1448)
Refs: #1371
2019-08-18 10:13:37 -07:00
Carl Lerche 88b4ec84d7 chore: prepare 0.2.0-alpha.2 release (#1465) 2019-08-17 23:34:25 -07:00
Philip Kannegaard Hayes 9f0daad5ac sync: fix fuzz_oneshot test by using instrumented loom::sync::Arc (#1464)
Since `tokio_sync::oneshot` makes a `CausalCell::with_mut()` mutable
access in the `Inner::drop()`, we must use the instrumented
`loom::sync::Arc`.

Uncovered by carllerche/loom#42
2019-08-17 21:30:31 -07:00
Carl Lerche c187cd75b6 signal: move into tokio-net (#1463) 2019-08-17 13:43:55 -07:00
Carl Lerche a83f5e4ba6 uds: move into tokio-net (#1462) 2019-08-16 14:42:05 -07:00
Carl Lerche 4935aae164 udp: remove files left over from moving tokio-udp (#1461) 2019-08-16 11:05:50 -07:00
Carl Lerche ba1829fd26 chore: rename ui-tests -> build-tests (#1460) 2019-08-16 09:26:56 -07:00
Carl Lerche ce7e60e396 udp: move tokio-udp into tokio-net (#1459) 2019-08-16 07:26:10 -07:00
Ivan Petkov d8b23ef852 signal: rename SignalKind methods (#1457)
This renames the SignalKind constructors to be a bit more readable
instead of using the signal names themselves
2019-08-15 21:09:09 -07:00
Carl Lerche 4788d3a9e3 tcp: move tokio-tcp into tokio-net (#1456) 2019-08-15 20:37:25 -07:00
Carl Lerche f1f61a3b15 net: reorganize crate in anticipation of #1264 (#1453)
Space is made to add `tcp`, `udp`, `uds`, ... modules.
2019-08-15 15:04:21 -07:00
Jakub Beránek d0a8e5d6f2 tokio-fs: rewrite std echo example using async/await (#1442)
This PR fixes the echo example in tokio-fs.

Refs: #1255
2019-08-15 13:10:17 -07:00
Carl Lerche 3b27dc31d2 threadpool: move threadpool into tokio-executor (#1452)
The threadpool is behind a feature flag.

Refs: #1264
2019-08-15 13:09:02 -07:00
Douman 37131b2114 runtime: refactor thread-local setters (#1449) 2019-08-15 13:00:57 -07:00
Carl Lerche 8538c25170 reactor: rename tokio-reactor -> tokio-net (#1450)
* reactor: rename tokio-reactor -> tokio-net

This is in preparation for #1264
2019-08-15 11:04:58 -07:00
Jakub Beránek 7b6438a172 tokio: rewrite print_each_packet example using async/await (#1446)
This PR fixes the print each packet example in tokio.

Refs: #1201
2019-08-15 10:42:34 -07:00
Carl Lerche 9de7083be8 executor: move current-thread into crate (#1447)
The `CurrentThread` executor is exposed using a feature flag.

Refs: #1264
2019-08-15 09:52:25 -07:00
John Doneth 8d55f98f6f udp: update tokio_udp::UdpFramed to std::future (#1370) 2019-08-14 11:18:21 -07:00
Taiki Endo 999a600494 io: add async BufReader/BufWriter (#1438) 2019-08-14 10:24:07 -07:00
Ilya Lakhin fb9809c068 executor, threadpool: forward port fix from #1155 (#1433)
Add executor::exit, allowing other executors inside threadpool::blocking.
2019-08-13 21:12:49 -07:00
Douman 517162792f macros: upgrade syn/quote (#1432) 2019-08-13 21:11:26 -07:00
Geoff Shannon fe90d61446 test: add a block_on function to tokio-test (#1431) 2019-08-13 21:10:26 -07:00
Ivan Petkov 338b37884a signal: Add SignalKind for registering signals more easily (#1430)
This avoids having consumers import libc for common signals, and it
improves discoverability since users need not be aware that libc
contains all supported constants.
2019-08-13 21:07:22 -07:00
Ivan Petkov 513326e01d signal: remove driver task for Windows event implementation (#1429)
Windows guarantees handler routines are always invoked in a new thread
(https://docs.microsoft.com/en-us/windows/console/handlerroutine), so we
don't need to use the handler-wake-another-driver technique used in the
Unix implementation

By broadcasting the event notifications from the handler, we no longer
need the Driver task to be spawned, which fixes the starvation issue if
the executor which runs the Driver task goes away

Also changed the behavior so that the default event handler runs if
all listeners for CTRL_{C, BREAK} events go away.
2019-08-13 21:01:06 -07:00
Ivan Petkov 73a91ad7b3 signal: delete blocking Read/Write impls on ChildStd{in, out, err} (#1428) 2019-08-13 20:53:02 -07:00
Taiki Endo 930cce8677 chore: update futures-preview to 0.3.0-alpha.18 (#1427) 2019-08-10 14:09:28 -07:00
Taiki Endo 6a125082e4 chore: apply unreachable_pub and missing_debug_implementations to all crates (#1424) 2019-08-11 04:28:52 +09:00
Taiki Endo d9f9c5658f chore: bump to newer nightly (#1426) 2019-08-11 02:01:20 +09:00
Taiki Endo fff39c03b1 ci: deny warnings in cirrus (#1425) 2019-08-11 01:41:51 +09:00
Tomasz Miąsko 756606a58b uds: implement split and split_mut for UnixStream (#1395)
This mirrors split API available in TcpStream.
2019-08-09 12:50:18 -07:00
Ran Benita e3b4c99a33 codec: a few suggestions (#1418)
How the buffer is managed is often critical for performance. Not
taking care of it will be catastrophic for performance beyond the
initial buffer size with the current implementation (a loop of
`reserve(1)`).
2019-08-09 12:20:31 -07:00
Taiki Endo 42fa0c28d3 timer: use std::sync::atomic::AtomicU64 instead of own AtomicU64 (#1421) 2019-08-10 03:42:03 +09:00
Taiki Endo f7b41c9dcc macros: improve error messages (#1420) 2019-08-09 10:28:22 -07:00
Taiki Endo 73102760cf chore: change default lint level to warning and deny warnings in CI (#1416) 2019-08-10 00:07:57 +09:00
Douman 18833a8e67 macros: Error on function with arguments (#1419) 2019-08-09 11:04:41 -04:00
tmiasko eba8bf2b4b io: implement AsyncWrite for Vec<u8> (#1409) 2019-08-08 20:55:27 -07:00
David Kellum 790d649dc5 update (dev dep) env_logger to latest 0.6 (#1390) 2019-08-08 20:37:32 -07:00
Lucio Franco 50e5d401df chore: prepare for v0.2.0-alpha.1 release (#1410) 2019-08-08 12:48:53 -07:00
Carl Lerche 2e69f2a7fd sync: track upstream loom changes (#1407) 2019-08-07 23:24:22 -07:00
Carl Lerche 962521f449 chore: enable full CI run (#1399)
* update all tests
* fix doc examples
* misc API tweaks
2019-08-07 20:02:13 -07:00
Carl Lerche 831be9c08e executor: remove unused dependency (#1406) 2019-08-07 19:55:42 -07:00
Carl Lerche 23c380a78f sync: track loom changes (#1405) 2019-08-07 15:38:34 -07:00
Lucio Franco 0a05332648 Remove git dep and add macro examples (#1404)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 15:02:38 -07:00
Lucio Franco 7268b0bb3a Migrate threadpool to futures-util (#1403)
* Migrate threadpool to futures-util

Signed-off-by: Lucio Franco <[email protected]>

* fmt
2019-08-07 16:26:44 -04:00
Lucio Franco 6412389bba executor: update park implementation (#1402)
Signed-off-by: Lucio Franco <[email protected]>
2019-08-07 13:01:13 -07:00
tmiasko 53a94c025d io: implement AsyncBufRead for &[u8] and Cursor (#1397)
* `impl AsyncRead for &[u8]`
* `impl AsyncBufRead for &[u8]`
* `impl<T: AsRef<[u8]> + Unpin> AsyncRead for Cursor<T>`
* `impl<T: AsRef<[u8]> + Unpin> AsyncBufRead for Cursor<T>`
2019-08-07 12:57:37 -07:00
Gurwinder Singh 7174c63bf9 codec: move length delimited codec to tokio-codec (#1401) 2019-08-07 12:10:05 -07:00
Ivan Petkov cb2336ff3d process: Misc polish (#1400)
* Denied all warnings in tests, and denied rust_2018_idioms violations
* Bumped the crate version and set publish = false
* Pruned dependencies:
 - Only pull in tokio-sync on windows where it is used
 - Removed unused dev-dependencies
* Switch to Async{Read, Write} traits from tokio-io rather than
futures-io
* Use #[tokio::test] where possible
* Removed deprecated items
* Fix all doc examples
2019-08-07 10:38:45 -07:00
Carl Lerche 47e2ff48d9 tokio: fix API doc examples (#1396) 2019-08-06 14:03:49 -07:00
Carl Lerche 2f43b0a023 sync: polish and update API doc examples (#1398)
- Remove `poll_*` fns from some of the sync types.
- Move `AtomicWaker` and `Lock` to the root of the `sync` crate.
2019-08-06 13:54:56 -07:00
Carl Lerche 05d00aebb7 uds: remove poll_* fns in favor of async fns (#1394) 2019-08-05 15:05:02 -07:00
Carl Lerche 62733a6594 udp: remove poll_* fns in favor of async fns (#1393)
This removes the need for manual futures.
2019-08-05 14:18:18 -07:00
Carl Lerche 6d8cc4e475 tcp: update API documentation (#1392) 2019-08-05 11:50:55 -07:00
Carl Lerche 6cbe3d4f82 fs: use async fn instead of custom futures (#1381)
Also update all the doc examples.
2019-08-04 11:24:30 -07:00
Carl Lerche 337646b97f tokio: re-export future/stream utils (#1387) 2019-08-03 21:08:29 -07:00
Taiki Endo 0bb015588a codec: add AsyncBufRead/BufRead implementations (#1385)
* AsyncBufRead for FramedWrite2<T>
* BufRead for FramedWrite2<T>
* AsyncBufRead for Fuse<T, U>
* BufRead for Fuse<T, U>
2019-08-03 20:15:50 -07:00
Steven Fackler 63377e2110 Add AsyncWriteExt::shutdown (#1382) 2019-08-03 00:51:24 -04:00
Carl Lerche 878503f965 docs: update API documentation for some crates (#1380)
Updates API documentation for

- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
2019-08-02 14:35:32 -07:00
Carl Lerche 2c01b3e0e0 io: remove util from default features (#1379)
Sub-crates should require opting into features.
2019-08-02 12:59:24 -07:00
Carl Lerche ee9105d166 tokio: add async io traits to prelude (#1378) 2019-08-02 12:50:40 -07:00
Lucio Franco 5a4f849bba tokio: update tinyhttp example to async/await (#1372) 2019-08-02 12:24:15 -07:00
Lucio Franco ff41108834 io: move io helpers back into tokio-io (#1377)
Utilities are made optional with a feature flag.
2019-08-02 12:23:44 -07:00
Lucio Franco 6b202722ea io: Add AsyncWriteExt::flush (#1376)
* io: Add `AsyncWriteExt::flush`

* fmt

* fix clippy
2019-08-02 13:53:49 -04:00
Lucio Franco 144d980e5c tokio: update connect to async/await (#1375) 2019-08-02 10:03:06 -07:00
Lucio Franco 81d789b88f tokio: Update proxy to async/await (#1373) 2019-08-01 20:03:34 -07:00
Lucio Franco 634c19582f chore: add rust-toolchain file to track nightly version (#1374) 2019-08-01 20:00:55 -07:00
Ivan Petkov ff922bbe6d signal: Change constructors to return a result instead of lazy future (#1340) 2019-07-30 18:23:26 -07:00
Gurwinder Singh bf38631d6a chore: Fix spelling mistake (#1359) 2019-07-30 10:53:11 -07:00
Taiki Endo 6dda866191 tokio: re-enable StreamExt (#1362) 2019-07-30 09:55:34 -07:00
Taiki Endo 03e450deb1 sync: switch branch of loom dev-dependency to master (#1367)
* sync: switch branch of loom dev-dependency to master

* replace loom::fuzz with loom::model
2019-07-30 10:11:46 -04:00
andy finch fbf90e6356 Update process to use std::future (#1343) 2019-07-29 18:36:11 -07:00
Shell Chen 74168ae82f tcp: add async fn TcpStream::peek (#1360)
* tcp: add `async fn TcpStream::peek`

* tcp: apply rustfmt on tests
2019-07-26 10:55:02 -04:00
John Doneth d038009e7d Update chat example to async/await (#1349) 2019-07-25 19:44:23 -04:00
John Doneth 132e9f1da5 Update examples to return Result (#1305)
* update echo-udp

* update echo

* update hello_world

* update udp-client

* rustfmt

* remove send & sync

* rebase & change new updated examples
2019-07-25 16:47:31 -04:00
Taiki Endo fe021e6c00 ci: enable clippy lints (#1335) 2019-07-26 03:47:14 +09:00
Lucio Franco f311ac3d4f buf: Inital pass at updating BufStream (#1355) 2019-07-25 14:21:48 -04:00
Shell Chen 298be80249 tokio: include async-trait feature for uds (#1352) 2019-07-25 08:07:33 -07:00
John Doneth 79b017c773 Export LinesCodecError (#1350) 2019-07-24 15:26:41 -04:00
Taiki Endo ca0e5cc670 add TryFrom/From implementations (#1347)
* TryFrom<net::TcpListener> for TcpListener
* TryFrom<net::TcpStream> for TcpStream
* TryFrom<net::UdpSocket> for UdpSocket
* TryFrom<net::UnixDatagram> for UnixDatagram
* TryFrom<net::UnixListener> for UnixListener
* TryFrom<net::UnixStream> for UnixStream
* TryFrom<UnixDatagram> for mio_uds::UnixDatagram
* TryFrom<File> for io::File
* From<io::File> for File
2019-07-24 09:26:12 -07:00
Douman 59bc364a0e macros: detect double test attribute (#1336) 2019-07-22 09:28:07 -07:00
Taiki Endo e88d10a3cb chore: bump to newer nightly (#1338) 2019-07-22 06:04:02 +09:00
Ivan Petkov a3b8d82711 Merge tokio-process into tokio
Original repo can be found at https://github.com/alexcrichton/tokio-process/
2019-07-21 11:08:16 -07:00
Ivan Petkov d9688bc094 signal: change unix::Signal to return () instead of signum (#1330)
* This simplifies the API surface by returning () instead of the signal
number that was used during registration. This also more closely mirrors
the cross-platform `CtrlC` event stream API
* This is a **breaking change**
2019-07-20 15:12:53 -07:00
Ivan Petkov 320a5fdca7 signal: replace windows::Event with windows::CtrlBreak (#1331)
* Add a new `windows::CtrlBreak` struct which wil represent a stream of
CTRL_BREAK_EVENT signals on Windows systems
* The `windows::Event` type is no longer publicly accessible and is
replaced by using `CtrlC` or `windows::CtrlBreak`.

[breaking-change]
2019-07-20 10:50:27 -07:00
Taiki Endo 9af07ce208 chore: remove redundant field names in struct literals (#1334) 2019-07-20 10:43:19 -07:00
Taiki Endo 1b2d997863 chore: use ptr::{null, null_mut} instead of 0 as *{const, mut} (#1333) 2019-07-20 10:41:02 -07:00
Taiki Endo 7a52ddcd09 chore: remove unnecessary conversion (#1332) 2019-07-20 12:10:42 -04:00
Carl Lerche 9d3e5aac08 tokio: remove Send + 'static requirement from block_on (#1329)
Removes the `Send` requirement to futures passed to `Runtime::block_on`.
Previously, `block_on` was implemented by sending the future to a
runtime thread. In order to do this, the future must be Send.

The reason why the future is sent to the pool is because we cannot
guarantee, while off the pool, that a reactor / timer thread is running.
This is due to a limitation in the current version of tokio-threadpool.
There is a plan to fix this (#1177), but the proper fix is non trivial.

In order to unblock APIs that require this, this patch updates the
runtime to spawn an always running thread containing a reactor and
timer. All calls to `block_on` will use that reactor and timer.
2019-07-19 17:25:04 -07:00
Carl Lerche a99fa6e096 chore: remove tokio-futures facade crate (#1327)
This switches from using the tokio-futures facade to referencing
futures-* crates directly.
2019-07-19 13:11:46 -07:00
David Kellum b89ed00a0d Remove last non-dev dependency on rand crate (#1324)
Use std RandomState for XorShift seeding. This allows dropping _rand_
crate dep here, accept as a dev dependency for tests or benchmarks.
2019-07-19 12:12:32 -07:00
Dylan Frankland 12ce75f088 fs: add remove_dir_all and RemoveDirAllFuture (#1325)
Adds the sister function to `remove_dir` and mirrors the `create_dir_all` that's already exposed.
2019-07-19 12:09:53 -07:00
Taiki Endo a88308ed9f tokio: add AsyncReadExt::read_to_string (#1326) 2019-07-19 11:50:00 -07:00
João Oliveira a298472da8 tokio-tls: enable Send and Sync (#1317)
-  update 0 as *mut () calls to std::ptr::null_mut()
- impl Send and Sync for AllowStd
2019-07-19 10:21:26 -07:00
Shell Chen d0bb16192b timer: change Into to From trait for Elapsed (#1322) 2019-07-17 09:04:59 -04:00
Shell Chen a18ddb3b61 timer: impl Into<std::io::Error> for Elpased (#1321)
That convert Elpased to ErrorKind::TimedOut
2019-07-16 20:22:03 -07:00
Jon Gjengset 003b4d8074 Get rid of Enter for with_default (#1315)
We want executors to enforce that there are never multiple active at the
same time. This is ensured through `Enter`, which will panic if you
attempt to create more than one. However, by requiring you to pass an
`&mut Enter` to `executor::with_default`, we were *also* disallowing
temporarily overriding the current executor.

This patch removes that requirement.
2019-07-16 14:29:35 -04:00
Yin Guanhao 6d186fe40e Replace (some) uninitialized with MaybeUninit (#1295) 2019-07-16 10:47:46 -07:00
Diggory Blake 0d99ddd4f4 tcp: implement "split_mut" for TcpStream (#1289) 2019-07-16 10:28:00 -07:00
João Oliveira 448d9d2eab tls: update to std-future (#1224) 2019-07-16 10:26:08 -07:00
David Kellum 0de3a69eb4 fs: drop deprecated tempdir crate use in tests (#1312)
In particular because it pulls in old rand duplicates. Replace use
with tempfile::tempdir() which has been available since tempfile
3.0.0.
2019-07-15 15:32:33 -07:00
John Doneth 61aee5fc28 examples: pdate tinydb example (#1288)
Update tinydb example to use async / await.
2019-07-15 15:19:36 -07:00
Sean McArthur 7f7f74985e io: Minor adjustments to tokio-test IO (#1306)
This also re-exports `bytes::{Buf, BufMut}` from `tokio-io`.
2019-07-15 14:53:16 -07:00
Jon Gjengset e6cf976662 tokio: include async-traits feature (#1314)
The `tokio` facade crate will depend on the `async-traits` feature flag in
sub crates.
2019-07-15 14:02:14 -07:00
Taiki Endo b14e189e44 add #[must_use] to more futures and streams (#1309) 2019-07-15 13:28:56 -07:00
Taiki Endo 2dde2b448f Fix import of ready macro 2019-07-15 11:52:13 -07:00
Taiki Endo 6742816e78 tokio: add AsyncBufReadExt::lines 2019-07-15 11:52:13 -07:00
Taiki Endo ab040bb498 tokio: add AsyncBufReadExt::read_line 2019-07-15 11:52:13 -07:00
Taiki Endo 0cfa120ba8 tokio: add AsyncBufReadExt::read_until 2019-07-15 11:52:13 -07:00
Taiki Endo 5774a9cd64 io: add AsyncBufRead trait 2019-07-15 11:52:13 -07:00
John Doneth da49ede41e update udp-codec example (#1293) 2019-07-15 14:14:03 -04:00
Carl Lerche d224d6415e chore: indicate the master branch docs are old. (#1304)
Fixes #1292
2019-07-15 10:44:47 -07:00
Gurwinder Singh 83273b8b50 chore: use ready macro from futures-core (#1300) 2019-07-15 10:43:54 -07:00
Taiki Endo ca708d6d87 chore: update rand dependency to 0.7 (#1302) 2019-07-15 10:13:10 -07:00
matthieugras 0b75c0c53d executor: block thread when needed in block fn (#1303)
Fix #1296
2019-07-15 08:56:20 -07:00
Alex Gaynor 5fbb36a060 reactor: bump parking_lot dependency (#1298) 2019-07-14 09:28:54 -07:00
Gurwinder Singh c897a5b696 Re-export tokio-fs (#1287) 2019-07-14 12:03:49 -04:00
Sean McArthur 48d7f7b931 tokio-test: add tokio_test::io mock builder 2019-07-12 11:19:12 -07:00
Carl Lerche 2291823181 tokio: re-export correct tokio-uds version (#1286)
An earlier PR (#1282) re-exported the version from crates.io and not git
master.
2019-07-11 10:27:51 -07:00
andy finch 795e02f4c6 fs: update to use std::future (#1269) 2019-07-11 09:05:49 -07:00
Carl Lerche 7ac8bfc821 chore: bump to newer nightly (#1284) 2019-07-10 14:36:36 -07:00
Carl Lerche a79483750f tokio: update echo example (#1283) 2019-07-10 14:21:20 -07:00
Carl Lerche 3855f373d3 tokio: re-export tokio-uds (#1282)
The tokio-uds crate has been previously updated to std::future. This
commit enables the re-export in the tokio facade crate.
2019-07-10 11:21:27 -07:00
Carl Lerche bd3f3270db tokio: update threaded runtime to std::future (#1280)
re-enables the threaded runtime and sets it (again) as the default.
2019-07-10 11:21:06 -07:00
Taiki Endo e5525628cd chore: remove usage of deprecated ONCE_INIT (#1281) 2019-07-10 08:35:07 -07:00
Carl Lerche f1b8a318d9 tokio: add AsyncReadExt::read_to_end (#1279) 2019-07-09 16:17:58 -07:00
Carl Lerche 64343f1b78 tokio: add AsyncWriteExt::write_all (#1277) 2019-07-09 12:37:14 -07:00
Ruben De Smet 82795184c1 tokio: rewrite examples with async. (#1228) 2019-07-09 11:21:12 -07:00
Thomas Lacroix f529928d87 chore: script updating versions in links to docs.rs (#1249) 2019-07-09 11:19:38 -07:00
Ivan Petkov 461eebe612 signal: Replace ctrl_c with a CtrlC struct (#1273)
* Add a new `CtrlC` struct which will represent a stream of SIGINT
signals on Unix or the CTRL_C event on Windows
* `CtrlC` implements `Stream<Output = ()>` rather than `IoSteam` as
previously
2019-07-09 08:48:46 -07:00
Gurwinder Singh 407d15cf93 chore: Add link to docs (#1276) 2019-07-09 11:21:29 -04:00
Yin Guanhao 80915906d8 uds: update to std-future (#1227) 2019-07-08 14:58:40 -07:00
Yin Guanhao 88e775dcf0 udp: UdpSocket split support (#1226) 2019-07-08 14:47:31 -07:00
Carl Lerche 8b49a1e05f chore: update examples link in README (#1274) 2019-07-08 13:34:39 -07:00
Carl Lerche 8fa1510d67 timer: fix build (#1275) 2019-07-08 11:23:15 -07:00
Reto Kaiser 7797a377c3 current-thread: make tokio_current_thread::Handle Sync (#1119) 2019-07-08 10:25:34 -07:00
Steven Fackler b62d224fac timer: fix Handle::timeout (#1093)
The old implementation didn't work for Timeout<Stream>, since the method
took a deadline rather than a timeout.
2019-07-08 10:18:37 -07:00
Aaron Hill d4803bc868 Use Sink trait from futures-sink-preview (#1244) 2019-07-08 09:56:11 -07:00
Thomas Lacroix e07a03b3c5 signal: update instructions in Ctrl-C example (#1270)
Fixes: #1248
2019-07-07 09:49:19 -07:00
Taiki Endo 7b86acb71d chore: Update futures-preview to 0.3.0-alpha.17 (#1267) 2019-07-04 14:34:57 -07:00
Steffen Butzer 0651f09427 Remove usage of deprecated std::error::Error methods (#1206) (#1245) 2019-07-03 23:06:03 -07:00
Thomas Lacroix 516251052d Add missing links in README.md (#1233)
Fixes: #1229
2019-07-03 22:59:10 -07:00
Ivan Petkov cbad83f362 signal: migrate to std::futures (#1218)
Migrate to std::futures and the futures 0.3 preview and use async/await
where possible

**Breaking change:** the IoFuture and IoStream definitions used to refer
to Box<dyn Future> and Box<dyn Stream>, but now they are defined as
Pin<...> versions which are technically breaking.

No other breaking or functional changes have been made
2019-07-03 10:40:59 -07:00
Eliza Weisman bd9760e124 add release documentation to CONTRIBUTING.md (#1171)
## Motivation

Currently, the process for releasing a new version of a Tokio crate is
somewhat complex, and is not well-documented. To make it easier for
contributors to release minor versions more frequently, there should be
documentation describing this process.

## Solution

This branch adds a section to `CONTRIBUTING.md` describing how to
release a new version of a Tokio crate. The steps are based on those
described by @carllerche in an offline conversation.

I've also added a quick shell script to actually publish new crate 
versions. This should make it harder to make mistakes when 
publishing.

Signed-off-by: Eliza Weisman <[email protected]>
2019-07-03 10:18:02 -07:00
Carl Lerche 3e898f58a5 tcp: add ascyc fn TcpListener::accept (#1242)
Refs: #1209
2019-07-03 09:49:56 -07:00
Ivan Petkov c531865d2c ci: don't generate docs for deps on FreeBSD (#1241) 2019-07-03 09:41:35 -07:00
Ivan Petkov 722eb257be ci: scope each tests/examples invocation to a specific crate (#1238) 2019-07-03 08:49:05 -07:00
Taiki Endo ceed29586b io: fix documents (#1231) 2019-07-01 20:44:12 -07:00
Carl Lerche 70eca184f0 tokio: re-enable timer in runtimes (#1237)
This also brings back the timer tests in the tokio crate.
2019-07-01 18:27:13 -07:00
Carl Lerche b2c777846e timer: finish updating timer (#1222)
* timer: restructure feature flags
* update timer tests
* Add `async-traits` to CI

This also disables a buggy `threadpool` test. This test should be fixed in the future.

Refs #1225
2019-06-30 08:48:53 -07:00
Lucio Franco 8e7d8af588 docs: add note in the readme about the master branch (#1230) 2019-06-29 21:47:20 -04:00
Yin Guanhao 7380dd2482 TcpSocket specialized split (#1217) 2019-06-28 23:36:49 -07:00
Eliza Weisman af46eac583 chore: remove tokio-trace, add "Related Projects" to README (#1221)
## Motivation

The `tokio-trace` and `tokio-trace-core` crates have been renamed to
`tracing` and `tracing-core`, and moved to their own repository
(`tokio-rs/tracing`).

## Solution

This branch removes `tokio-trace` and `tokio-trace-core` from the
`tokio` repository. In addition, I've added a "Related Projects" section
to the root README, which lists `tracing` (as well as  `mio`, and
`bytes`) as other libraries maintained by the Tokio project. I thought
that this would help folks looking for `tokio-trace` here find it in its
new home.

In addition, it changes `tokio` to depend on `tracing-core` rather than
`tokio-trace-core`.

Closes #1159

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-28 13:13:46 -07:00
Carl Lerche e7488d983e threadpool: update to std::future (#1219)
An initial pass at updating `tokio-threadpool` to `std::future`. The
codebase and tests both now run using `std::future` but the wake
mechanism is not ideal. Follow up work will be required to improve on
this.

Refs: #1200
2019-06-27 22:30:56 -07:00
Sean McArthur e4415d986a sync: change oneshot poll_close to poll_closed
The action of `Sender::poll_close` is to check if the receiver has been
closed, not to try to close the sender itself. So change to
`poll_closed`.
2019-06-27 13:56:58 -07:00
Carl Lerche ff906acdfb ci: disable cache on cirrus (#1215)
Caching takes longer than rebuilding
2019-06-27 12:08:43 -07:00
Carl Lerche 32ceccb465 sync: add async APIs to oneshot and mpsc (#1211)
Adds:

- oneshot::Sender::close
- mpsc::Receiver::recv
- mpsc::Sender::send

Also renames `poll_next` to `poll_recv`.

Refs: #1210
2019-06-27 11:33:36 -07:00
Douman 0af05e7408 macros: allow configuring runtime used by main macro (#1185) 2019-06-27 10:40:21 -07:00
jesskfullwood 6b9e7bdace codec: update to use std-future (#1214)
Strategy was to

- copy the old codec code that was temporarily being stashed in `tokio-io`
- modify all the type signatures to use Pin, as literal a translation as possible
- fix up the tests likewise

This is intended just to get things compiling and passing tests. Beyond that there is surely
lots of refactoring that can be done to make things more idiomatic. The docs are unchanged.

Closes #1189
2019-06-27 10:10:29 -07:00
Carl Lerche ed4d4a5353 chore: format code and enable rustfmt CI task (#1212) 2019-06-27 00:05:01 -07:00
Carl Lerche 1f47ed3dcc tokio: rewrite io_read.rs test to use async/await (#1207)
This simplifies the test
2019-06-26 17:06:56 -07:00
Carl Lerche e9aaacddbd tokio: re-export sync::{lock,mpsc} (#1208)
These types have been updated already.
2019-06-26 16:54:15 -07:00
Carl Lerche 11f6b2862f tokio: move I/O helpers to ext traits (#1204)
Refs: #1203
2019-06-26 14:42:19 -07:00
Carl Lerche 8404f796ac test: get cargo test --tests working (#1205)
Broken tests are disabled
2019-06-26 14:40:52 -07:00
Yin Guanhao 6316aa1d0b Update tokio-udp to use std-future (#1199) 2019-06-26 14:41:36 -04:00
Bhargav 0784dc2767 tokio: add read_exact method (#1202) 2019-06-26 11:36:09 -07:00
Denis dd126c2333 Implement TryFrom to transform various I/O primitives into their mio counterparts (#1158)
* `TryFrom<TcpListener> for mio::net::TcpListener`
* `TryFrom<TcpStream> for mio::net::TcpStream`
* `TryFrom<UdpSocket> for mio::net::UdpSocket`
* `TryFrom<UnixListener> for mio_uds::UnixListener`
* `TryFrom<UnixStream> for mio_uds::UnixStream`
2019-06-26 08:51:38 -07:00
Lucio Franco 3cc33dca7c sync: Fix lock test to actually test the inner lock value (#1197)
* sync: Fix lock test to actually test the returned value

* Update lock test to use task.poll
2019-06-26 11:32:41 -04:00
Carl Lerche dc5fa80a09 macros: re-export main macro from tokio (#1198)
Includes minor fixes and a very basic example.

Fixes #1183
2019-06-25 20:14:21 -07:00
Zahari Dichev 455782b964 trace: Allow setting event parents explicitly (#1109)
## Motivation 

As mentioned in tokio-rs/tracing#1100  it makes sense to be able to set
the parents of events explicitly.

## Solution 

For that to happen the Parent type is extracted from span.rs and a
`parent` field is added to Event. Additionally the appropriate macros
arms are added with corresponding tests as described in
tokio-rs/tracing#1100

Closes tokio-rs/tracing#1100

Signed-off-by: Zahari Dichev <[email protected]>
2019-06-25 15:12:52 -07:00
Lucio Franco 29e417c257 tokio: Add io copy, read, and write (#1187) 2019-06-25 16:51:49 -04:00
Ivan Petkov 9df1140340 signal: factor out event delivery into its own module to share between Unix and Windows (#1174)
Today the Unix and Windows implementations have similar yet differing
implementations of hooking into OS events and propagating them to any
listening futures. Rather than re-implement the same behavior two
different ways, we should factor out any commonality into a shared
module and keep the Unix/Windows modules focused solely on OS
integrations.

Reusing the same implementation across OS versions also allows for more
consistent behavior between platforms, which also makes squashing bugs
much easier.

This change introduces the `registry` module which handles creating and
initializing a global map of signals/events and their registered
listeners. Each OS specific module is expected to implement the OS hooks
which delegate to invoking the registry module's methods for
distributing the event notifications.

# Use registry module for Windows implementation

Note this still uses the same architecture as previously: a driver task
is spawned by the first registered event, and that task is responsible
for delivering any events to registered futures. (If that first event
loop goes away, all events will deadlock). A solution to this issue will
be explored at a later time.
2019-06-25 13:07:59 -07:00
Lucio Franco e2b4bdb647 sync: Add LockFuture for Lock (#1184) 2019-06-25 10:42:35 -07:00
Ivan Petkov c6defbce4b process: Move files to their own directory 2019-06-24 17:31:47 -07:00
Ivan Petkov b7846a4e2f process: Remove unneeded files 2019-06-24 17:31:00 -07:00
Ivan Petkov cb8607a816 process: Update to 2018 edition 2019-06-24 17:29:33 -07:00
Ivan Petkov 27c15471c1 process: Run cargo fmt 2019-06-24 17:29:33 -07:00
Ivan Petkov 0ab25878bd process: Update README 2019-06-24 17:29:32 -07:00
Eliza Weisman 448302c3d4 trace: Improve documentation (#1148) 2019-06-24 19:22:05 -05:00
Ivan Petkov 934a1467d4 process: Update CHANGELOG 2019-06-24 17:12:17 -07:00
Ivan Petkov 4d639e246b process: Update Cargo.toml 2019-06-24 17:10:58 -07:00
Ivan Petkov ff5381de8d process: Update license files 2019-06-24 17:10:58 -07:00
Ivan Petkov 061452dc01 process: Delete flaky and (now) unused test 2019-06-24 17:10:58 -07:00
Ivan Petkov a6b2682309 process: Bump to 0.2.4 2019-06-24 16:57:20 -07:00
Ivan Petkov cf84a59e5a process: Don't kill child on drop if already successfully killed 2019-06-24 16:57:20 -07:00
Ivan Petkov e90e33d5df process: Add unit tests for dropping killing dropped children 2019-06-24 16:57:20 -07:00
Ivan Petkov fa5da27d98 process: Utilize a global orphan process queue to avoid leaks 2019-06-24 16:57:20 -07:00
Ivan Petkov ecaa069f0f process: Implement a queue for repeatedly attempting to reap orphaned processes 2019-06-24 16:57:20 -07:00
Ivan Petkov fc15d7d4a4 process: Only pull in mio dependency on unix platforms 2019-06-24 16:57:20 -07:00
Ivan Petkov a70a3b599a process: ci: move cargo tool installation to after_success 2019-06-24 16:57:20 -07:00
Ivan Petkov 26faefcc34 process: ci: enable clippy checks as part of the build 2019-06-24 16:57:19 -07:00
Ivan Petkov f16725ea9f process: Fix clippy warnings 2019-06-24 16:57:19 -07:00
Ivan Petkov caf43221b5 process: ci: fix cargo binary caching 2019-06-24 16:57:19 -07:00
Ivan Petkov 93680357dd process: Fix drop_kills test when running on macOS with a single thread 2019-06-24 16:57:19 -07:00
Ivan Petkov 784d21ae31 process: Try pinning mio to 0.1.16 2019-06-24 16:57:19 -07:00
Ivan Petkov 0938ccfefd process: ci: cache cargo tarpaulin build 2019-06-24 16:57:19 -07:00
Ivan Petkov 6fa2fdab44 process: Ensure all tests are run with an explicit timeout 2019-06-24 16:57:19 -07:00
Ivan Petkov d0d13d0bd0 process: Change codecov comment behavior to default 2019-06-24 16:57:19 -07:00
Ivan Petkov 8a1777b800 process: Rename EventedReaper to Reaper 2019-06-24 16:57:18 -07:00
Ivan Petkov 42d0f53ddb process: Optimize out the "reaped" flag 2019-06-24 16:57:18 -07:00
Ivan Petkov db0c4147c8 process: Refactor Unix process handling 2019-06-24 16:57:18 -07:00
Ivan Petkov 10fd2afd18 process: Simplify child IO registration 2019-06-24 16:57:18 -07:00
Ivan Petkov 83a55601ef process: Move src/unix.rs to src/unix/mod.rs 2019-06-24 16:57:18 -07:00
Ivan Petkov b37120f61c process: Update line-by-line doc example to be more flexible 2019-06-24 16:57:18 -07:00
Ivan Petkov 91dbf24cf4 process: Update min supported rust version as per the Tokio project policy 2019-06-24 16:57:18 -07:00
Ivan Petkov c78fd6d6c5 process: Update Travis link from .org to .com 2019-06-24 16:57:18 -07:00
Ivan Petkov 025474dfbb process: ci: Install cargo-tarpaulin *after* initial tests 2019-06-24 16:57:18 -07:00
Ivan Petkov e7dfcf90fe process: ci: Enable code coverage tracking via codecov.io 2019-06-24 16:57:17 -07:00
Ivan Petkov ecdfe4c474 process: ci: collect code coverage info via cargo-tarpaulin 2019-06-24 16:57:17 -07:00
Ivan Petkov 37b4efb9e2 process: Bump version to 0.2.3 2019-06-24 16:57:17 -07:00
Ivan Petkov c94f607f1b process: Fix some test case deprecation warnings 2019-06-24 16:57:17 -07:00
Ivan Petkov 76438c9e70 process: Implement AsRawHandle for ChildStd{in, out, err} for parity 2019-06-24 16:57:17 -07:00
Yuya Nishihara e0e9594f71 process: Implement AsRawFd for ChildStd* structs 2019-06-24 16:57:17 -07:00
Yuya Nishihara 3b43262a10 process: Implement AsRawFd for inner Fd<T> wrappers and use it instead of self.0 2019-06-24 16:57:17 -07:00
Ivan Petkov 5f18bf669f process: Bump minimum supported rustc version to 1.26 2019-06-24 16:57:17 -07:00
Ivan Petkov 1581c8b475 process: Bump minimum required version of tokio-signal to 0.2.5 2019-06-24 16:57:16 -07:00
Ivan Petkov d3b2efc815 process: Add regression test for signal starvation 2019-06-24 16:56:53 -07:00
Ivan Petkov f7c4e3cd84 process: Bump min supported rustc version to 1.25 2019-06-24 16:56:53 -07:00
Ivan Petkov 329ad3324c process: Bump to 0.2.2 2019-06-24 16:56:53 -07:00
Ivan Petkov 2b6695d25a process: Update CHANGELOG 2019-06-24 16:56:53 -07:00
Ivan Petkov 9290602815 process: Unix: preregister for signal notifications before polling child 2019-06-24 16:56:53 -07:00
Ivan Petkov 827e77e71e process: Bump to 0.2.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 7b3e4b98ac process: Update Child::forget example to use the tokio runtime 2019-06-24 16:56:52 -07:00
Ivan Petkov 5e9d60e834 process: Add a CHANGELOG 2019-06-24 16:56:52 -07:00
Ivan Petkov 8270965459 process: Remove dependency on tokio-core 2019-06-24 16:56:52 -07:00
Ivan Petkov e6b044a820 process: Bump tokio-signal version to 0.2 2019-06-24 16:56:52 -07:00
Ivan Petkov de9b401457 process: Mark status_async2/StatusAsync2 as deprecated 2019-06-24 16:56:52 -07:00
Ivan Petkov ad5179b2d5 process: Remove all items deprecated in 0.1 2019-06-24 16:56:52 -07:00
Ivan Petkov 0aceba21bd process: Bump to 0.1.6 2019-06-24 16:56:52 -07:00
Ivan Petkov 09e21eceea process: Unix: mark child as reaped on kill 2019-06-24 16:56:52 -07:00
Arvid E. Picciani bdc87856f2 process: fix zombification on Drop on unix 2019-06-24 16:56:51 -07:00
Ivan Petkov 7987b64445 process: Clarify that Child::forget docs that it can leak OS resources 2019-06-24 16:56:51 -07:00
Alex Crichton 32c928b607 process: Bump to 0.1.5 2019-06-24 16:56:51 -07:00
Alex Crichton 82aeae147d process: Update dev-dependencies 2019-06-24 16:56:51 -07:00
Alex Crichton f48944c1fb process: Update winapi to 0.3 2019-06-24 16:56:51 -07:00
Ivan Petkov f0680617ee process: Fix project name typo in README 2019-06-24 16:56:51 -07:00
Alex Crichton dbc185cd3a process: Tweak travis config 2019-06-24 16:56:51 -07:00
Alex Crichton c205e2c358 process: Fix copy/paste 2019-06-24 16:56:51 -07:00
Alex Crichton acec6356ee process: Clarify wording of license information in README. 2019-06-24 16:56:51 -07:00
Alex Crichton c11eec3908 process: Bump to 0.1.4 2019-06-24 16:56:50 -07:00
Alex Crichton 69295fac1e process: Add an Errors section to status_async2 2019-06-24 16:56:50 -07:00
Ivan Petkov b9c6eb309c process: Add status_async2 as a closer analog to spawn_async 2019-06-24 16:56:50 -07:00
Ivan Petkov 56d3914675 process: Bugfix: ensure status_async closes child's stdio handles after spawning 2019-06-24 16:56:50 -07:00
Ivan Petkov 34e71fa71a process: Add must_use annotations to all futures 2019-06-24 16:56:50 -07:00
Ivan Petkov 914b803429 process: Add Debug impls for nondeprecated structs 2019-06-24 16:56:50 -07:00
Alex Crichton 4d11784b01 process: Tweak docs and macro imports 2019-06-24 16:56:50 -07:00
Michael Pankov 50cabae181 process: Add an example with reading input line-by-line 2019-06-24 16:56:50 -07:00
Alex Crichton c101e9e11d process: Use appveyor to download rustup 2019-06-24 16:56:49 -07:00
Alex Crichton 5c5f793ef0 process: Bump to 0.1.3 2019-06-24 16:56:49 -07:00
Alex Crichton 1384b31d60 process: Update to tokio-io, mio, and tokio-core changes 2019-06-24 16:56:49 -07:00
Alex Crichton 521dc94021 process: Bump to 0.1.2 2019-06-24 16:56:49 -07:00
Alex Crichton ed23a06fb1 process: Update doc urls and metadata 2019-06-24 16:56:49 -07:00
Alex Crichton 1aee22505a process: Remove caveat about tokio-signal 2019-06-24 16:56:49 -07:00
Alex Crichton 01b5bf6761 process: Use join3 instead of two joins 2019-06-24 16:56:49 -07:00
Alex Crichton 6638cbc80e process: Update README 2019-06-24 16:56:49 -07:00
Alex Crichton 22bc5e2738 process: Bump back to 0.1.1 2019-06-24 16:56:49 -07:00
Alex Crichton a0c162c0ff process: Hide compat from docs 2019-06-24 16:56:48 -07:00
Alex Crichton ca51ae9651 process: Add back in 0.1.0 compatibility layer 2019-06-24 16:56:48 -07:00
Alex Crichton f3f99b723f process: Bump to 0.2.0 2019-06-24 16:56:48 -07:00
Alex Crichton f20e7a4d2b process: Bump minimum version of tokio-core 2019-06-24 16:56:48 -07:00
Alex Crichton 4a92c4d4b6 process: Tweak drop_kills test 2019-06-24 16:56:48 -07:00
Alex Crichton 9680ecc109 process: Share init in tests 2019-06-24 16:56:48 -07:00
Alex Crichton 124391e42b process: Add a simple wait_with_output test 2019-06-24 16:56:48 -07:00
Alex Crichton 6150be189f process: Rewrite the crate with an extension trait 2019-06-24 16:56:48 -07:00
Ivan Petkov ca9586a089 process: Add documentation to public declarations 2019-06-24 16:56:47 -07:00
Ivan Petkov 89b9792931 process: Update README with crates.io info 2019-06-24 16:56:47 -07:00
Alex Crichton a0cc60153a process: Fix nightly tests 2019-06-24 16:56:47 -07:00
Alex Crichton 7f3f868b66 process: Add Windows support for stdio streams 2019-06-24 16:56:47 -07:00
Andreas Rottmann 97ebb2275c process: [WIP] Actually be non-blocking 2019-06-24 16:56:47 -07:00
Andreas Rottmann 849a5ad0b2 process: Add support for stdio streams 2019-06-24 16:56:47 -07:00
Alex Crichton b16a8613b1 process: Test on stable 2019-06-24 16:56:47 -07:00
Alex Crichton 5664660156 process: Fix tests on nightly 2019-06-24 16:56:47 -07:00
Alex Crichton 4416ea07d8 process: Update travis token 2019-06-24 16:56:47 -07:00
Alex Crichton 56222c588b process: pass --target on appveyor 2019-06-24 16:56:46 -07:00
Alex Crichton 72179d49c5 process: Update to crates.io versions of deps 2019-06-24 16:56:46 -07:00
Alex Crichton 073a1a251a process: Track tokio-core master 2019-06-24 16:56:46 -07:00
Alex Crichton 31c81faf96 process: Add appveyor to readme 2019-06-24 16:56:46 -07:00
Alex Crichton 5e68b0d51d process: Don't build on stable, start w/ beta for now 2019-06-24 16:56:46 -07:00
Alex Crichton 4bd07ac6aa process: Add metadata info 2019-06-24 16:56:46 -07:00
Alex Crichton f4f7bb232e process: Fix a test on Windows 2019-06-24 16:56:46 -07:00
Alex Crichton 413e1b78a7 process: Fix a segfault on windows 2019-06-24 16:56:46 -07:00
Alex Crichton 649fa13a15 process: Remove unused imports 2019-06-24 16:56:45 -07:00
Alex Crichton eef655f3b1 process: Add a Windows implementation 2019-06-24 16:56:45 -07:00
Alex Crichton 97508096fa process: Initial commit 2019-06-24 16:56:41 -07:00
Carl Lerche 06c473e628 Update Tokio to use std::future. (#1120)
A first pass at updating Tokio to use `std::future`.

Implementations of `Future` from the futures crate are updated to implement
`Future` from std. Implementations of `Stream` are moved to a feature flag.

This commits disables a number of crates that have not yet been updated.
2019-06-24 12:34:30 -07:00
James Gilles aa99950b9c trace: Switch benchmarks to criterion (#1163)
Extracted from #1152

This makes it possible to run benchmarks on stable + gives more statistical reliability.
2019-06-24 12:05:45 -07:00
Takanori Ishibashi aac6998c22 chore: fix url in docs (#1173) 2019-06-24 07:33:46 -04:00
Matt Bilker df2c3cd475 trace: fix debug and debug_span macro regression from #1103 (#1170)
PR #1103 accidentally changed the log level for the debug and
debug_span macros to use the INFO level instead of the DEBUG
level. This PR corrects this regression back to the intended
behavior.
2019-06-22 16:31:25 -07:00
James Gilles 36ed35c52c trace: add program-wide default dispatcher (#1152)
## Motivation

I was just trying to use tokio-trace for a greenfield project, but I was frustrated to discover that I couldn't really use it easily.

I was using the [`runtime`](https://docs.rs/runtime/0.3.0-alpha.4/runtime/) crate, which transparently spawns a thread pool executor for futures. In that thread pool, there's no way to set a tokio-trace subscriber for the duration of each thread, since you don't control the thread initialization. You *might* be able to wrap every future you spawn with a subscriber call, but that's a lot of work.

I was also confused because the documentation said that setting a subscriber in the main thread would use that subscriber for the rest of the program. That isn't the case, though -- the subscriber will be used only on the main thread, and not on worker threads, etc.

## Solution

I added a function `set_global_default`, which works similarly to the `log` crate:

```rust
tokio_trace::subscriber::set_global_default(FooSubscriber::new());
```

The global subscriber (actually a global `Dispatch`) is a `static mut` protected by an atomic; implementation is copied from the `log` crate. It is used as a fallback if a thread has no `Dispatch` currently set. This is extremely simple to use, and doesn't break any existing functionality.

Performance-wise, thread-local `Dispatch` lookup goes from ~4.5ns to ~5ns, according to the benchmarks. So, barely any runtime overhead. (Presumably there's a little compile-time overhead but idk how to measure that.) Since the atomic guard is only ever written once, it will be shared among a CPU's cores and read very cheaply.

I added some docs to partially address #1151. I also switched the tokio-trace benchmarks to criterion because the nightly benchmarks weren't compiling (missing `dyn` flags?)
2019-06-21 16:49:53 -07:00
Eliza Weisman 5925ca7720 trace: fix level_span macros not propagating parents (#1167)
Currently, when the `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!`, and `error_span!` macros are invoked with an explicit
parent, a name, and zero or more fields (no target), the macros don't
pass along the explicitly provided parent when expanding to the `span!`
macro. This is likely due to an oversight on my part.

This branch fixes these macros by adding the parent into the `span!`
macro expansion. I've also added a test to catch regressions

Shoutout to @jonhoo for catching this one!

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-21 11:17:06 -07:00
Max Bruckner 2ac132fb46 runtime: better error message in block_on_all on panics (#1166) 2019-06-21 11:10:58 -04:00
Hung-I Wang f9a0cb8792 timer: Implement Default for DelayQueue (#1118) 2019-06-21 10:42:52 -04:00
Igor Gnatenko 9fa6092e5a chore: Update parking_lot to 0.8 (#1078) 2019-06-21 10:42:09 -04:00
Eliza Weisman d4adeeef2f trace: Remove the AsId trait (#1145)
While we're making breaking changes to `tokio-trace`, it would be good
to get rid of the `AsId` trait. The goal of span functions that are
generic over `Span`/`Id` can be achieved without the unnecessary
complexity of defining a new trait. This would also make the API added
to `tokio_trace_core::Event` in #1109 more consistent with the
`tokio-trace::Span` API.

This branch removes `AsId` from `tokio-trace` and replaces its uses with
`impl Into<Option<Id>>` and `impl Into<Option<&'a Id>>`. While `AsRef`
might be more semantically correct for the borrowed-`Id` conversion, its
signature doesn't permit conversion into an `Option`. Implementations of
`Into<Option<Id>>` and `Into<Option<&'a Id>>` have been added for 
`tokio_trace::Span`.

This is _technically_ a breaking API change, as it changes function
signatures. However, the existing macro syntax still works as-is, and
the tests which pass `&Id`, `&Span`, and `&Option<Id>` to the span
macros all still compile after this change.

Closes #1143

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-13 12:53:08 -07:00
Steven Fackler 4f6395b31c Make threadpool::Runtime methods take &self (#1140)
The runtime is inherently multi-threaded, so it's going to have to deal
with synchronization when submitting new tasks anyway. This allows a
runtime to be shared by multiple threads more easily when e.g. building
a blocking facade over a tokio-based API.
2019-06-10 12:54:27 -07:00
yanjhk 5c0b56278b Use ThreadPool's impl of spawn (#1139) 2019-06-10 11:23:12 -07:00
Eliza Weisman 41ca9a43de trace: Add shorthand syntax for local fields (#1103)
## Motivation

A common pattern in `tokio-trace` is to use the value of a local
variable as a field on a span or event. Currently, this requires code
like:
```rust
info!(foo = foo);
```
which is not particularly ergonomic given how commonly this occurs.
Struct initializers support a shorthand syntax for fields where the name
of the field is the same as a local variable, and `tokio-trace` should
as well.

## Solution

This branch adds support for syntax like
```rust
let foo = ...;
info!(foo);
```
and 
```rust
let foo = Foo {
    bar: ...,
    ...
};
info!(foo.bar)
```
to the `tokio-trace` span and event macros. This syntax also works with
the `Debug` and `Display` field shorthand.

The span macros previously used a field name with no value to indicate 
an uninitialized field. A new issue, #1138, has been opened for finding a
replacement syntax for uninitialized fields. Until then, the `tokio-trace` 
macros will no longer provide a way to create fields without values, 
although the `-core` API will continue to support this.

Closes #1062 

Signed-off-by: Eliza Weisman <[email protected]>
2019-06-09 13:16:35 -07:00
Carl Lerche 8d0f102de8 Merge branch 'v0.1.x' into merge-0.1 2019-06-05 12:28:39 -07:00
Kevin Leimkuhler 5dcb379f6d Bump tokio-sync to 0.1.6 (#1123) 2019-06-05 12:19:06 -07:00
Kevin Leimkuhler 970f75f830 sync: Add Sync impl for Lock (#1117) 2019-06-04 17:04:35 -07:00
Kevin Leimkuhler 619efed28b sync: Add Sync impl for Lock (#1116)
Signed-off-by: Kevin Leimkuhler <[email protected]>
2019-06-03 11:12:28 -07:00
Carl Lerche 18ed0be851 executor: remove unnecessary APIs from Enter. (#1115) 2019-05-31 11:11:10 -07:00
Carl Lerche 01052f930a Bump tokio version to v0.1.21. (#1113) 2019-05-30 14:39:30 -07:00
Lucio Franco 940f2c3431 Update tokio-trace-core to 0.2 (#1111)
Also includes 1b498e8aa2
2019-05-30 11:33:55 -07:00
Eliza Weisman 84d5a7f5a0 trace: Change Span::enter to return a guard, add Span::in_scope (#1076)
## Motivation

Currently, the primary way to use a span is to use `.enter` and pass a
closure to be executed under the span. While that is convenient in many
settings, it also comes with two decently inconvenient drawbacks:

 - It breaks control flow statements like `return`, `?`, `break`, and
   `continue`
 - It require re-indenting a potentially large chunk of code if you wish
   it to appear under a span

## Solution

This branch changes the `Span::enter` function to return a scope guard 
that exits the span when dropped, as in:
```rust
let guard = span.enter();

// code here is within the span

drop(guard);

// code here is no longer within the span
```
The method previously called `enter`, which takes a closure and 
executes it in the span's context, is now called `Span::in_scope`, and
was reimplemented on top of the new `enter` method. 

This is a breaking change to `tokio-trace` that will be part of the
upcoming 0.2 release.

Closes #1075 

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-24 15:24:13 -07:00
Carl Lerche 1b498e8aa2 Fix TCP poll_hup test (#1106)
This updates tests to track a fix applied in Mio. Previously, Mio
incorrectly fired HUP events. This was due to Mio mapping `RDHUP` to
HUP. The test is updated to correctly generate a HUP event.

Additionally, HUP events will be removed from all platforms except for
Linux. This is caused by the inability to reliably map kqueue events to
the epoll HUP behavior.
2019-05-24 14:08:07 -07:00
Eliza Weisman b2c53987d9 trace: Add shorthand for field::display and field::debug (#1088)
## Motivation

In `tokio-trace`, field values may be recorded as either a subset of
Rust primitive types or as `fmt::Display` and `fmt::Debug`
implementations. Currently, `tokio-trace` provides the `field::display`
and `field::debug` functions which wrap a type with a type that
implements `Value` using the wrapped type's `fmt::Display` or
`fmt::Debug` implementation. However, importing and using these
functions adds unnecessary boilerplate. 

In #1081, @jonhoo suggested adding shorthand syntax to the macros,
similar to that used by the `slog` crate, as a solution for the
wordiness of the current API.

## Solution

This branch adds `?` and `%` sigils to field values in the span and
event macros, which expand to the `field::debug` and `field::display`
wrappers, respectively. The shorthand sigils may be used in any position
where the macros take a field value.

For example:
```rust
trace_span!("foo", my_field = ?something, ...); // shorthand for `debug`
info!(foo = %value, bar = false, ...) // shorthand for `display`
```

Adding this shorthand required a fairly large change to how field
key-value pairs are handled by the macros --- since `%foo` and `%foo`
are not valid Rust expressions, we can no longer match repeated 
`$ident = $expr` patterns, and must now match field lists as repeated
token trees. The inner helper macros for constructing `FieldSet`s and
`ValueSet`s have to parse the token trees recursively. This added a
decent chunk of complexity, but fortunately we have a large number of
compile tests for the macros and I'm quite confident that all existing
invocations will still work.

Closes #1081

Signed-off-by: Eliza Weisman <[email protected]>
2019-05-21 10:31:48 -07:00
Carl Lerche 38092010c4 Merge branch 'v0.1.x' 2019-05-14 11:50:44 -07:00
Carl Lerche 475dabe96d Release tokio v0.1.20, tokio-timer v0.2.21, and remove async-await-preview feature. (#1089)
The `async-await-preview` feature is removed as 0.1 will no longer track
Rust nightly.

This also bumps:
- tokio-timer (0.2.11).
2019-05-14 11:21:24 -07:00
Carl Lerche cb4aea394e Update Tokio to Rust 2018 (#1082) 2019-05-14 10:27:36 -07:00
Jeehoon Kang 79d8820050 Fix link in tokio-futures/README.md (#1085)
`tokio-futures/README.md`'s link to the examples was wrong.
2019-05-10 10:19:32 -07:00
Carl Lerche 951f2fd910 test: re-export macro dependencies (#1077)
Callers may not always have `futures` available at the root of the
crate. Re-exporting dependencies makes them available to the macro at a
deterministic location.
2019-05-03 20:43:40 -07:00
Carl Lerche 4ef736b9d5 async-await: add current_thread::Runtime::block_on_async (#1072)
This function is used by the Tokio macros introduced by #1058  but was
omitted from the PR.
2019-04-30 19:55:22 -07:00
Steven Fackler 219f24cbf1 timer: Replace Handle::deadline with Handle::timeout (#1074)
Deadline was deprecated a while ago and replaced with Timeout, but the
methods on Handle got missed.

Fixes #1071
2019-04-30 10:29:54 -07:00
Carl Lerche ea282efb2e ci: fix isRelease condition (#1066) 2019-04-29 10:37:00 -07:00
Michal 'vorner' Vaner 042224d33c signal: Smaller dependency (#1069)
The signal-hook library got split into lower-level and higher-level
parts. The tokio-signal uses only API from the lower-level one, so it
can depend on it directly.

The only effect of this change is smaller amount of compiled (and
unused) code during compilation. There's no change in the code actually
used.
2019-04-28 19:12:40 -07:00
Ian Hamlin 927eb80ad4 Fix an error in the mit-url in the README.md (#1068) 2019-04-27 12:56:40 -07:00
Carl Lerche 6a8934e897 Fix threadpool dependency (#1061) 2019-04-25 22:23:24 -04:00
Carl Lerche 0e400af78c Async/await polish (#1058)
A general refresh of Tokio's experimental async / await support.
2019-04-25 22:22:32 -04:00
Carl Lerche df702130d6 tcp: fix some tests that spuriously fail (#1060)
This does not remove all cases of using a fixed port in doc tests, but
removing some should reduce the likelihood of spurious failures.
2019-04-25 12:01:39 -07:00
Carl Lerche 949adbb887 chore: remember to remove path deps on release (#1057) 2019-04-24 10:42:39 -07:00
Ryan Dahl b2b796a228 rt: forward panic_handler to tokio::runtime::Builder (#1055) 2019-04-24 10:41:42 -07:00
Igor Gnatenko abb014efc2 tokio: Bump min version of tokio-sync (#1054)
It is needed for lock functionality which tokio now uses.
2019-04-24 08:24:59 -07:00
Lucio Franco e5cf0cc717 Introduce tokio-test crate (#1030) 2019-04-23 20:17:57 -07:00
Carl Lerche 62f34e15ce Bump tokio to 0.1.19. (#1053)
This also bumps:

- tokio-async-await (0.1.7)
- tokio-buf (0.1.1)
- tokio-sync (0.1.5)
- tokio-threadpool (0.1.14)
2019-04-22 15:12:25 -07:00
Eliza Weisman 3ebca76a9a trace: prepare tokio-trace for release (#1051) 2019-04-22 14:15:07 -07:00
Ryan Dahl fea1f780bc threadpool: add panic_handler (#1052) 2019-04-21 16:26:09 -07:00
Eliza Weisman 712ca84033 trace-core: prepare for 0.2 release (#1047) 2019-04-21 10:20:28 -07:00
Jon Gjengset cf06621998 tokio-sync: Add async mutual exclusion primitive (#964)
This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.

This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
2019-04-18 13:16:26 -04:00
Lucio Franco 7e51ab05e9 buf: Add IntoStream (#1048)
* buf: Add IntoStream

* Add debug implementation for IntoStream

* Add get_ref, get_mut and into_inner
2019-04-18 11:42:34 -04:00
Eliza Weisman 4bfa4ffcdf trace-core: Dispatchers unset themselves (#1033)
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans. 

Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.

Before:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          51 ns/iter (+/- 12)
test span_repeatedly         ... bench:       5,073 ns/iter (+/- 1,528)
test span_with_fields        ... bench:          56 ns/iter (+/- 49)
test span_with_fields_record ... bench:         363 ns/iter (+/- 61)
```

After:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          35 ns/iter (+/- 12)
test span_repeatedly         ... bench:       4,165 ns/iter (+/- 298)
test span_with_fields        ... bench:          48 ns/iter (+/- 12)
test span_with_fields_record ... bench:         363 ns/iter (+/- 91)
```

Closes #1032 

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-16 15:51:45 -07:00
Taiki Endo 88b942652c async-await: fix examples (#1050)
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
  on Rust 2018
* Fix an example in the documentation
2019-04-16 14:19:38 -04:00
Taiki Endo 5029e80a89 async-await: update to new futures_api (#1049) 2019-04-16 14:07:03 -04:00
Eliza Weismanandcsmoe 847fb59b17 trace-core: Introduce callsite classification in metadata (#1046)
## Motivation

To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.

## Solution

A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.

`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.

New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.

Fixes: #986
Closes: #1008

Co-Authored-By: csmoe <[email protected]>
2019-04-11 11:56:42 -07:00
Jane Lusby b4fe517a16 trace-core: add a function to rebuild cached interest (#1039)
## Motivation

Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.

However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`.  Even when filters change very
infrequently, they must still always be re-evaluated every time.

In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.

## Solution

This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.

## Breaking Change

In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.

Closes #1038

Co-Authored-By: yaahallo <[email protected]>
2019-04-10 13:51:05 -07:00
Simon Wollwage 7ae010f0f3 async-await: Use Context instead of Waker in poll (#1041)
Rust nightly std::future::Future recently changed Waker
to Context.

Change to use Context

Co-Authored-By: Kintaro <[email protected]>
2019-04-10 09:14:31 -07:00
Carl Lerche 9144b2ff53 sync: remove unnecessary imports (#1043) 2019-04-09 12:26:11 -07:00
Lev Eniseev 2c4549a18a Add example of blocking environment (#1036) 2019-04-09 12:10:15 -07:00
João Oliveira 4f819b7cd1 trace: fix counters example, Span IDs must be greater than zero (#1037)
## Motivation
tokio-trace counter example was panicking due to returning 0 as the first Span Id

## Solution
start ID's with 1 instead
2019-04-05 13:51:20 -07:00
Eliza Weisman 197f88f3bc trace: Change Span::enter and record to take &self (#1029)
## Motivation

The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.

Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
    span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
    span.record("bar" &false);
});
```

Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.

The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.

## Solution

This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows. 

In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`. 

## Notes

In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.

Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.

Before:
```
test enter_span              ... bench:          13 ns/iter (+/- 1)
```

...and after:
```
test enter_span              ... bench:           3 ns/iter (+/- 1)
```

Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.

I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 15:06:47 -07:00
Eliza Weisman 44f65afcc6 trace: Allow field names to be separated by .s (#1027)
## Motivation

In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.

## Solution

This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.

## Notes

The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.

Closes #1018.
Closes #1022.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 14:44:11 -07:00
Eliza Weisman 4271a9cd8d trace: Fix subscriber benchmarks panicking (#1028)
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-02 13:42:31 -07:00
Matthias Prechtl 9d8096b911 Improve documentation of Subscriber::record and Subscriber::event (#1026) 2019-04-02 12:44:01 -07:00
João Oliveira 597f271c08 trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation 

Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952). 

Closes #1013

## Solution 

Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels

## Notes 

I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
2019-04-02 11:29:23 -07:00
Taiki Endo 599955f716 Replace try! macro with ? operator (#1024) 2019-04-01 13:45:59 -07:00
Ivan Petkov 91bb0f73f5 signal: refactor Windows registrations to be lazy (#1001)
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
  so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
  when calling `global_init`

Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS

Fixes #999
2019-04-01 12:46:22 -07:00
Eliza Weisman 6c9d8abba9 trace: Make Span API functions taking IDs a little more flexible (#1021)
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.

Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-01 11:42:29 -07:00
Carl Lerche 824b7b6759 buf: stream and iter helpers (#1011) 2019-03-29 12:26:13 -07:00
Carl Lerche cb91dd274a buf: impl Error for CollectVecError (#1010) 2019-03-29 08:49:08 -07:00
Eliza Weisman ea7178b8c6 trace-core: Add slightly more useful debug impls (#1014)
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.

When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.

The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-28 14:47:06 -07:00
Carl Lerche a99b8e2e0b buf: impl size_hint for str types + reorg tests (#1012) 2019-03-28 14:02:45 -07:00
Carl Lerche 03859a7dcd buf: implement FromBufStream for Bytes (#1009) 2019-03-27 19:22:06 -07:00
Eliza Weisman d8177f81ac trace: Allow trace instrumentation to emit log records (#992)
## Motivation

`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.

Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.

## Solution

This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation. 

## Notes

In order to allow spans to log their names when they are entered and 
exited even when the span is disabled, this branch adds an 
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.

Fixes: #949

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-26 16:43:05 -07:00
Son ceca2a3cd6 chore: add license to tokio (#1006) 2019-03-26 08:41:41 -07:00
Son 1524ee4b60 trace: Add static level filtering (#987)
## Motivation

`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.

## Solution

We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.

Closes #959
2019-03-25 15:19:20 -07:00
MOZGIII 7793d63739 Corrected doc for tokio_buf SizeHint (#1003) 2019-03-25 16:54:49 -04:00
Red Hara e0e26bc223 Fix typo in README.md in examples (#1002) 2019-03-24 13:02:11 -04:00
Eliza Weisman 9c5cad037f trace-core: Add overrideable downcasting to Subscribers (#974)
## Motivation

In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to 
downcast the current `Dispatch` to a concrete subscriber type.

For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust

pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
    tokio_trace::dispatcher::with(|c| {
        if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
            s.do_elizatracing_correlation_magic(tt, et);
        }
    }); 
}
```

This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).

## Solution

This branch makes the following changes:
 * Add an object-safe `downcast_raw` method to the `Subscriber` trait,
   taking a `TypeId` and returning an `*const ()` if the type ID 
   matches the subscriber's type ID, or `None` if it does not, and
 * Add `is<T>` and `downcast_ref<T>` functions to `Subscriber` 
   and `Dispatch`, using `downcast_raw`.

Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_ 
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour 
somehow, can downcast to the inner type if they choose to.

[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204

Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 16:21:46 -07:00
Eliza Weisman 30330da11a chore: Fix examples not working with cargo run (#998)
* chore: Fix examples not working with `cargo run`

## Motivation

PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:

```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
error: no example target named `chat`

Did you mean `echo`?
```

## Solution

This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:

```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
   Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
   Compiling tokio-reactor v0.1.9
   Compiling tokio-threadpool v0.1.13
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.10
   Compiling tokio-uds v0.2.5
   Compiling tokio-udp v0.1.3
   Compiling tokio-tcp v0.1.3
   Compiling tokio-fs v0.1.6
   Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
    Finished dev [unoptimized + debuginfo] target(s) in 7.04s
     Running `target/debug/examples/chat`
server running on localhost:6142
```

Signed-off-by: Eliza Weisman <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 15:25:42 -07:00
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
Carl Lerche 11e2af66a8 Bump Tokio to v0.1.15. (#869)
Also bumps:

- tokio-sync (0.1.0)
- tokio-threadpool (0.1.11)
- tokio-timer (0.2.9)
2019-01-25 10:20:09 -08:00
Carl Lerche a4aae1459c chore: move enumerate test to correct location (#867) 2019-01-24 20:47:45 -08:00
Zahari Dichev 12546d1d9c tokio-timer: fix DelayQueue bug when inserting shorter delay (#863)
Reset the delay of the queue in case an item that expires sooner than the last inserted is put
into the queue.
2019-01-24 14:36:41 -08:00
Zahari Dichev fbad6297c5 Add enumerate combinator to Stream (#832) 2019-01-24 11:50:34 -08:00
Jon Gjengset 0ec8986b0b Make reason for try_send errors clearer (#864) 2019-01-23 15:06:24 -08:00
Jon Gjengset c6f8bdb249 Remove T: Debug bound on mpsc Debug impls (#866)
Following from https://github.com/tokio-rs/tokio/pull/865, this PR
removes `#[derive(Debug)]` on `mpsc` sender and receiver types in favor
of explicit `impl fmt::Debug` blocks that don't have a `T: fmt::Debug`
bound.
2019-01-23 18:04:00 -05:00
Jon Gjengset c6f9a069a5 Explicit impl Clone for tx to avoid T: Clone (#865)
`#[derive(Clone)]` on a type `struct Foo<T>` adds an impl that requires that
`T: Clone`:

```rust
impl<T: Clone> Clone for Foo<T>
```

which is unfortunate in the case of senders, because we don't want to require
that the items being sent are `Clone` for the channel sender to be `Clone`.
This PR adds an explicit `impl Clone` for the bounded and unbounded sender
types which does not have the `T: Clone` bound.

Note that this is _also_ an issue with `#[derive(Debug)]`, but that one is
harder to work around as `chan::Tx` _also_ has `#[derive(Debug)]`, as does
`chan::Chan`, so we'd have to add explicit impls for all of them to make
progress.
2019-01-23 15:51:44 -05:00
Sean McArthur 9f356d6244 tokio-sync: add into_inner for TrySendErrors (#862) 2019-01-22 14:48:22 -08:00
Carl Lerche 13083153aa Introduce tokio-sync crate containing synchronization primitives. (#839)
Introduce a tokio-sync crate containing useful synchronization primitives for programs
written using Tokio.

The initial release contains:

* An mpsc channel
* A oneshot channel
* A semaphore implementation
* An `AtomicTask` primitive.

The `oneshot` and `mpsc` channels are new implementations providing improved
performance characteristics. In some benchmarks, the new mpsc channel shows
up to 7x improvement over the version provided by the `futures` crate. Unfortunately,
the `oneshot` implementation only provides a slight performance improvement as it
is mostly limited by the `futures` 0.1 task system. Once updated to the `std` version
of `Future` (currently nightly only), much greater performance improvements should
be achievable by `oneshot`.

Additionally, he implementations provided here are checked using
[Loom](http://github.com/carllerche/loom/), which provides greater confidence of
correctness.
2019-01-22 11:37:26 -08:00
rmcteggart-r7 91f20e33a4 docs: deal with Result instead of using unwrap (#860) 2019-01-20 14:21:17 -05:00
Eliza Weisman 983e9d1b67 timer: Fix DelayQueue delay reset logic (#851) 2019-01-20 08:38:39 -05:00
Stjepan Glavina 4c8f274db9 threadpool: drop incomplete tasks on shutdown (#722)
## Motivation

When the thread pool shuts down, futures that have been polled at least once but not completed yet are simply leaked. We should drop them instead.

## Solution

Multiple changes are introduced:

* Tasks are assigned a home worker the first time they are polled.

* Each worker contains a set of tasks (`Arc<Task>`) it is home to. When a task is assigned a home worker, it is registered in that worker's set of tasks. When the task is completed, it is unregistered from the set.

* When the thread pool shuts down and after all worker threads stop, the remaining tasks in workers' sets are aborted, i.e. they are switched to the `Aborted` state and their `Future`s are dropped.

* The thread pool shutdown process is refactored to make it more robust. We don't  track the number of active threads manually anymore. Instead, there's  `Arc<ShutdownTrigger>` that aborts remaining tasks and completes the `Shutdown` future once it gets destroyed (when all `Worker`s and `ThreadPool` get dropped because they're the only ones to contain strong references to the `ShutdownTrigger`).

Closes #424 
Closes #428
2019-01-17 22:12:25 +01:00
Marek Kotewicz c980837581 docs: missing links in tokio-timer::delay_queue (#845) 2019-01-13 21:20:08 +01:00
Marek Kotewicz eec370cae8 docs: fixed links in tokio-timer (#844)
* docs: fixed links in tokio-timer/src/timer/mod.rs

* docs: fixed links in tokio-timer::clock
2019-01-12 10:06:55 -08:00
Marek Kotewicz 733d432b80 docs: fixed links to tokio_timer::clock::Now (#842)
* docs: fixed links to tokio_timer::clock::Now in tokio-timer/src/timer/mod.rs

* docs: fixed links to std::time::Instant in tokio-timer/src/timer/mod.rs
2019-01-10 23:49:13 +01:00
Carl Lerche 74c473d68f travis: allow nightly Rust CI to fail (#843) 2019-01-10 11:27:58 -08:00
Sean McArthur d95c697781 tokio: update tokio-threadpool minimum version (#838) 2019-01-07 16:49:08 -08:00
Carl Lerche 25e835c5b7 tcp: specify version for tokio dev dependency
This is required for publishing to crates.io
2019-01-06 23:31:24 -08:00
Carl Lerche 961aae41c4 Bump version to 0.1.14. (#836)
Also bumps:

* tokio-async-await (0.1.5)
* tokio-executor (0.1.6)
* tokio-fs (0.1.5)
* tokio-io (0.1.11)
* tokio-reactor (0.1.8)
* tokio-tcp (0.1.3)
* tokio-threadpool (0.1.10)
* tokio-tls (0.2.1)
* tokio-uds (0.2.5)

...and updates LICENSE files to 2019.
2019-01-06 23:25:55 -08:00
Carl Lerche 74c73b218e Revert "util: implement stream debounce combinator (#747)" (#834)
This reverts commit 7a49ebb65e.

The commit conflicted with another change that was merged, causing CI to fail. The public API
also requires a bit more refinement (#833) and Tokio crates need to be released.
2019-01-06 16:56:49 -08:00
Moritz Gunz 7a49ebb65e util: implement stream debounce combinator (#747) 2019-01-05 11:08:12 -05:00
Stjepan Glavina a687922746 tcp: deprecate TcpStream::try_clone() (#824) 2019-01-05 10:55:58 -05:00
Stjepan Glavina df299ced45 threadpool: panic if a worker thread cannot be spawned (#826) 2019-01-05 10:53:38 -05:00
Carl Lerche 78d1fe0eb0 ci: limit min rust version to cargo check (#829) 2019-01-05 10:52:26 -05:00
Ryan Huang fc8cde383a docs: fix link to ThreadPool (#830) 2019-01-05 10:51:17 -05:00
Sean McArthur 76198f63d7 Provide optional features on tokio crate (#808)
Disabling all features means the only dependency is `futures`.

Relevant pieces of the API can then be enabled with the following features:

- `codec`
- `fs`
- `io`
- `reactor`
- `tcp`
- `timer`
- `udp`
- `uds`

This also introduces the beginnings of enabling only certain pieces of the `Runtime`. As a start, the entire default runtime API is enabled via the `rt-full` feature.
2019-01-04 11:42:33 -08:00
Carl Lerche 39dc5706b7 travis: remove commented out code. (#828)
The commented out lines are no longer relevant and will not be brought
back.
2019-01-03 22:04:09 -08:00
Carl Lerche cbecb87797 executor: fix build (#825)
Two unrelated PRs to the same file resulted in a broken build. This
patch fixes the build by including `Arc`.
2019-01-03 11:26:58 -08:00
Carl Lerche f0bdf1980c threadpool: remove unused fn (#822)
The unused lint on nightly has discovered a new unused fn.
2019-01-03 09:34:37 -08:00
Stjepan Glavina 5e2d93f060 Use Crossbeam's Parker/Unparker (#528) 2019-01-02 21:51:22 -08:00
Taiki Endo 9a8d087c69 Allow deprecated Error::cause (#818)
Error::cause is deprecated in Rust 1.33, but this allows Error::cause
until the minimum supported version of tokio is Rust 1.30.

When the minimum support version of tokio reaches Rust 1.30,
replace Error::cause with Error::source.

Fixes: #817
2019-01-02 14:12:11 -08:00
gralpli 30f59670c8 Clarify what NoopWaker does (#819) 2019-01-02 12:29:30 -08:00
jq-rs 9e4ddaeaf3 examples: single-threaded chat combinator example (#794) 2018-12-29 10:16:30 -05:00
Balthild Ires 03e2e864f3 Stablize pin feature (#814)
Box::pinned has been renamed to Box::pin. Meanwhile, the pin feature
no longer requires an attribute to enable.

Fixes: #813
2018-12-28 12:12:31 -08:00
Sean McArthur c8a990eda4 tokio-reactor: deprecates Handle::current() (#805)
The side effects of calling `Handle::current()` from outside of a
runtime could be very surprising, since it would start up a background
reactor.
2018-12-28 12:09:35 -08:00
Pavel Strakhov 1a5026324f executor: impl Unpark for Arc<Unpark> (#802) 2018-12-28 14:40:04 -05:00
Stjepan Glavina fdf4aba621 threadpool: introduce a global task queue (#798) 2018-12-28 14:34:54 -05:00
Roman 201b6ce53a ci: remove ALLOW_FAILURES=false in travis for nightly cargo doc (#816) 2018-12-28 10:06:00 -05:00
Roman db69275202 docs: fix warnings for nightly docs (#792) 2018-12-17 15:20:46 -05:00
Roman af85cb3430 ci: improve travis run times (#793) 2018-12-17 15:18:27 -05:00
Christian Bourjau 36f1a19ac8 Minor change in documentation of Decoder::decode (#797)
`None` -> `Ok(None)`
2018-12-13 11:14:38 -08:00
Stjepan Glavina 6aa990ea75 threadpool: fix semaphore deadlock (#795) 2018-12-12 16:42:18 -05:00
Simon Farnsworth 760a7667d6 threadpool: improve the documentation of blocking (#789) 2018-12-05 15:20:19 -05:00
Matt Gathu 2283b63e9e fs: added usage examples/doctests to File (#786) 2018-12-01 21:07:48 -05:00
Felix Obenhuber 8263e5f18d uds: fix WouldBlock case in UnixDatagram send methods (#782) 2018-11-30 19:40:03 -05:00
Carl Lerche b3e57b60d0 examples: remove reference to tokio-core (#780) 2018-11-28 14:55:31 -05:00
Matt Gathu 1cd0ebfc5e tcp: add usage examples to TcpListener and TcpStream (#775)
Refs: https://github.com/rust-lang-nursery/wg-net/issues/54
2018-11-28 13:05:35 -05:00
David Kellum 4797d79950 reactor: update to parking_lot 0.7 (#778) 2018-11-28 09:29:31 -08:00
Steven Fackler e7d9ba7e51 tls: make TlsConnector and TlsAcceptor derive Clone (#777) 2018-11-27 07:52:17 -05:00
luben karavelov 527dc0a66f net: export UnixDatagram and UnixDatagramFramed (#772) 2018-11-23 08:41:32 -05:00
Carl Lerche b117fc1d65 Bump version to v0.1.13 (#771)
This also bumps the following sub crate versions:

* tokio-current-thread (0.1.4)
* tokio-reactor (0.1.7)
* tokio-signal (0.2.7)
* tokio-threadpool (0.1.9)
* tokio-timer (0.2.8)
* tokio-udp (0.1.3)
* tokio-uds (0.2.4)
2018-11-21 17:11:31 -08:00
Felix Obenhuber 272e09d349 threadpool: remove smoke example (#764) (#770) 2018-11-21 14:23:36 -08:00
Stjepan Glavina 3235749006 threadpool: refactor pool shutdown (#769) 2018-11-20 21:43:23 +01:00
Stjepan Glavina 9c037044c4 threadpool: rename inner to something more descriptive (#768)
`inner` is a fitting name for variables of type named `Inner`, but in other cases I find them confusing - sometimes `inner` refers to a `Pool`, sometimes to a `Sender`. I renamed a bunch of variables named `inner` to be more descriptive.

This PR is the first step in an effort of splitting https://github.com/tokio-rs/tokio/pull/722#issuecomment-439552671 into multiple PRs.
2018-11-20 20:05:14 +01:00
Patrick Barrett 3658e10045 uds: implement UnixDatagramFramed (#453)
Implement `Stream + Sink` layer on top of unix domain sockets
using codecs.
2018-11-20 09:19:34 -08:00
Carl Lerche ed3ece266b current-thread: fix shutdown on idle (#763)
When spawning using `Handle` while on the executor, tasks were being
double counted. This prevented the number of active tasks to reach zero,
thus preventing the executor from shutting down.

This changes `spawn` to check if being called from the executor
**before** incrementing the number of active tasks.

Fixes #760
2018-11-20 09:17:07 -08:00
Liran Ringel 9b1a45cc6a tests: handle errors properly in examples (#748) 2018-11-20 11:10:36 -05:00
Carl Lerche 477fa5580a ci: Don't deploy docs if $TARGET is set (#762) 2018-11-19 21:22:28 -08:00
Toby Lawrence bb6cca8ff0 tests: switch to Windows Server 2016 for AppVeyor builds. (#761)
Should hopefully fix the underlying bug that was causing tokio-tls tests to occasionally fail on Windows.

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 20:18:37 -05:00
Moritz Gunz e166c4d912 Implement throttle combinator (#736)
Throttle down a stream by enforcing a fixed delay between items.
2018-11-19 15:04:55 -08:00
Toby Lawrence b7506cf663 Allow nightly builds to fail. (#743)
* tests: allow nightly builds to fail

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 17:13:56 -05:00
Carl Lerche dc4a29359f io: allow deprecated code in length_delimited test (#759)
This file is testing deprecated code, so it should be permitted to
access deprecated code.
2018-11-19 14:11:46 -08:00
Bastian Köcher d3dca4552b Expose after_start and before_stop in runtime::Builder (#756)
Closes #705
2018-11-19 09:04:58 -08:00
andoks 42a0df1ea4 Fix async await README example (#758)
* async-await: fix README example dependencies

As per commit "async-await: track nightly changes (#661)" ( commit
2f690d30bc)

> The `tokio-async-await` crate is no longer a facade. Instead, the
> `tokio` crate provides a feature flag to enable async/await support.

Ensure the example in the async-await README file also works by
correctly declaring this updated dependency

* async-await: remove unnecessary 'edition' declaration from README

As the "edition" feature was stabilized in rust v1.30 and async-await
specifies that the nightly toolchain must be used, remove the use of the
"edition" feature gate since it is enabled by default.
2018-11-17 20:58:19 -08:00
Brian Myers a98eab6eff rt: fix Builder docs to no longer use deprecated methods (#749) 2018-11-16 14:58:08 -08:00
Ivan Petkov 5a5dde70b3 signal: miscellaneous tweaks and improvements (#751)
* Minimize allocation needed for channels

* Use a newtype for signal ids

* We can just cast the raw pointer to a `usize` and still perform a
simple identity check, without incurring any implications of storing a
raw pointer (e.g. previously Signal was !Sync and had an unsafe impl of
Send, and now it is naturally Sync+Send)

* Broadcast with `try_send` instead of `start_send`

The `Stream::start_send` method uses backpressure and schedules the
current task to be notified whenever the channel has additional room,
which means we'll generate a lot of unnecessary wakeups whenever a
channel gets full

By changing to `try_send` and handling any errors, we ensure the
Driver's task won't get woken up when a Signal finally consumes its
notification, since we're coalescing things anyway
2018-11-16 14:56:43 -08:00
Alex Gaynor d0963774a3 chore: bump rand dependency to 0.6 (#753) 2018-11-16 14:54:14 -08:00
Felix Obenhuber c83355235c uds: minor doc fix in UnixStream and UnixDatagram (#754) 2018-11-16 14:53:19 -08:00
Kazuyoshi Kato 33a216e4c1 fs: add more tests (#755)
Fixes #704.
2018-11-16 14:50:06 -08:00
Toralf Wittner 09f2ac85bf udp: add into_parts to RecvDgram (#710)
* udp: add `into_parts` to `RecvDgram`

If `RecvDgram` can not be driven to completion it may become necessary to get back the `UdpSocket` it contains which is currently not possible.

This adds`into_parts` to get the socket as well as the buffer back. Both methods consume `RecvDgram`.

Note that after the future has completed, `into_parts` must not be used, or else a panic will happen.
2018-11-15 10:55:34 -05:00
Ohad Ravid 32a152630f uds: added solaris support in the ucred module (#733) 2018-11-15 10:30:37 -05:00
Kazuyoshi Kato 9153067d66 fs: add tests for directory-related functions (#704) (#724)
This change adds a few tests around directory-related functions.
2018-11-13 18:28:52 -05:00
Kazuyoshi Kato d246964bdf fs: gen_ascii_chars has been deprecated (#735)
Use sample_iter() instead.
2018-11-10 21:35:38 -05:00
Alex Gaynor e700607554 Bumped crossbeam-utils version (#746)
## Motivation

tokio depends on an out of date version of crossbeam-utils, which results in multiple versions of that package being linked in binaries which use other popular libraries.

## Solution

Bump the version; there's no API changes and tests still pass.
2018-11-10 10:39:09 +01:00
Benjamin Saunders 5321550534 Derive Clone for delay_queue::Key (#730)
Improves API ergonomics with minimal forwards-compatibility hazard.
2018-11-09 15:11:17 -08:00
Stjepan Glavina 32e1cafb57 fix tsan errors (#745) 2018-11-09 15:06:46 -08:00
Josh Leverette 49bc4025dd reactor: reduce log level of loop process (#734) 2018-11-07 16:53:53 -05:00
Carl Lerche 51e36e41bc Add tokio-buf and a BufStream trait (#611)
The `BufStream` trait provides an improved API for working with
asynchronous streams of bytes compared to `Stream<Item = [u8]>`
2018-10-29 13:43:48 -07:00
Carl Lerche d011b92b9a rt: fix Runtime::reactor() as used by tokio-core (#721)
* rt: fix `Runtime::reactor()` as used by tokio-core

Up until Tokio v0.1.11, the handle returned by `Runtime::reactor()`
pointed to a reactor instance running in a background thread. The thread
was eagerly spawned.

As of v0.1.12, a reactor instance is created per runtime worker thread.
`Runtime::reactor()` was deprecated and updated to point to the reactor
for one of the worker threads.

A problem occurs when attempting to use the reactor before spawning a
task. Worker threads are spawned lazily, which means that the reactor
referenced by `Runtime::reactor()` is not yet running.

This patch changes `Runtime::reactor` back to a dedicated reactor
running on a background thread. However, the background thread is now
spawned lazily when the deprecated function is first called.

Fixes #720

* Fix comment

Co-Authored-By: carllerche <[email protected]>
2018-10-25 11:23:54 +02:00
Carl Lerche f929576f0e Bump version to 0.1.12 (#718)
Also bumps the following sub-crates:

* tokio-fs (0.1.4)
* tokio-io (0.1.10)
* tokio-signal (0.2.6)
* tokio-threadpool (0.1.8)
* tokio-uds (0.2.3)
2018-10-23 22:00:49 -07:00
Ivan Petkov b0f001a05a signal: Bump version to 0.2.6 (#714)
* Also Update the CHANGELOG to match the rest of the project
2018-10-23 20:39:19 -07:00
Iku Iwasa 2291ba9d0d uds: add NetBSD support (#715) 2018-10-23 20:15:42 -04:00
Name 7f84f6b4ca contributing: fix an invalid link (#716)
Just move a dot to the right place.
2018-10-21 16:56:01 +00:00
Andrew Audibert 5f61bd5252 fix a typo in the contributing guide (#711) 2018-10-19 09:25:11 -07:00
Ryan Dahl bffa3ed558 fs: expose fs::File::from_std() (#696) 2018-10-17 19:51:46 -04:00
Sean McArthur 7b5ef61aeb runtime: check Enter in more places when blocking (#708)
- `tokio::run` checks Enter before creating a new threadpool and
  spawning the main future.
- `Runtime::block_on` now checks Enter
- `Runtime::block_on_all` now checks Enter
2018-10-17 15:25:40 -07:00
Stjepan Glavina 753336de8e threadpool: Arc instead of Inner in Notifier (#702) 2018-10-15 13:24:00 -07:00
Ryan Levick 65aea16ad1 tokio: change hello world to new, simpler example (#690) 2018-10-12 12:42:19 -04:00
nickelc 796fee6364 fs: fix minor documentation error for MetadataFuture (#698) 2018-10-12 12:41:19 -04:00
Stjepan Glavina adb0ba71d4 threadpool: worker threads shouldn't respect keep_alive (#692)
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.

Bug fixes and new features should include tests.

Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->

## Motivation

Now that each worker thread drives its own reactor, reactors have to be driven until the threadpool shuts down. We mustn't use the `keep_alive` setting to shut down a worker thread if it doesn't receive an event from the reactor for a certain duration of time.

<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->

## Solution

Just ignore the `keep_alive` setting when parking in `Worker::sleep`.

<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
2018-10-10 09:05:36 +02:00
David Ross bfa6766f3c re-export tokio_io::read in tokio::io (#689)
Fixes: #688
2018-10-09 19:50:03 -07:00
Nikolay Kim a2f457fa48 io: expose underlying codec (#686) 2018-10-06 19:23:05 -04:00
Eliza Weisman 1879bc49ce codec: Fix panic in LengthDelimitedCodec::encode (#682)
Fixes: #681 

## Motivation

Currently, a potential panic exists in `LengthDelimitedCodec::encode`.
Writing the length field to the `dst` buffer can exceed the buffer
capacity, as `BufMut::put_uint_{le,be}` doesn't reserve more capacity. 

## Solution

This branch adds a call to `dst.reserve` to ensure that there's 
sufficient remaining buffer capacity to hold the length field and
the frame, prior to writing the length field. Previously, capacity
was only reserved later in the function, when writing the frame
to the buffer, and we never reserved capacity for the length field.

I've also added a test that reproduces the issue. The test panics on
master, but passes after making this change.

Signed-off-by: Eliza Weisman <[email protected]>
2018-10-04 12:46:57 -07:00
Sven Marnach 678f6382b8 io: implement prepare_uninitialized_buffer for Take and Chain (#678) 2018-10-04 11:03:43 -07:00
Stjepan Glavina e27b0a46ba threadpool: spawn new tasks onto a random worker (#683)
* threadpool: submit new tasks to a random worker

* Revert unnecessary version bumps
2018-10-03 23:09:20 +02:00
Stjepan Glavina d35d0518f5 runtime: create reactor per worker (#660) 2018-10-02 18:19:27 -07:00
Sven Marnach 886511c0a6 io: fix minor documentation errors for Async{Read,Write} (#677) 2018-10-01 19:34:24 -04:00
Steven Fackler d06bd6b216 Expose keep_alive on the Runtime builder (#676)
This was overlooked when delegating the rest of the threadpool builder
methods from Runtime's builder.
2018-09-28 21:00:50 -07:00
Carl Lerche 2c85cd0991 Bump version to v0.1.11 (#675)
This fixes the dependency on `tokio-async-await` to not be scoped to
unix platforms.

Fixes #673
2018-09-28 11:32:52 -07:00
Carl Lerche 1e45237a28 Bump tokio-uds to v0.2.2 2018-09-27 20:05:23 -07:00
Sean McArthur 3a88d85538 ads: fix UdsStream::read_buf to clear read (not write) readiness (#672) 2018-09-27 20:02:29 -07:00
Carl Lerche b47ad24268 Bump version to v0.1.10, fixing minimal versions (#671)
Some minimal versions were not correctly updated.

Also updates:

* tokio-current-thread (v0.1.3).
2018-09-27 13:00:53 -07:00
Carl Lerche cab9a44e01 Bump version to v0.1.9 (#666)
This also includes bumps to subcrates.

* tokio-async-await (0.1.4)
* tokio-codec (0.1.1)
* tokio-current-thread (0.1.2)
* tokio-executor (0.1.5)
* tokio-io (0.1.9)
* tokio-reactor (0.1.6)
* tokio-tcp (0.1.2)
* tokio-threadpool (0.1.7)
* tokio-timer (0.2.7)
2018-09-26 22:32:51 -07:00
Carl Lerche 964afb2ce3 split facade modules into separate files (#665) 2018-09-26 15:28:57 -07:00
Carl Lerche 2f690d30bc async-await: track nightly changes (#661)
The `tokio-async-await` crate is no longer a facade. Instead, the `tokio` crate
provides a feature flag to enable async/await support.
2018-09-26 10:10:47 -07:00
Stjepan Glavina 331a88cee6 reactor: turn bench-poll into a proper benchmark (#662) 2018-09-26 08:52:35 -07:00
Andrew Tunnell-Jones 46353737e7 tinydb: Update doc to reflect change from RefCell to Mutex (#663)
Fixes: #658
2018-09-26 11:47:31 +02:00
Carl Lerche ffd73a64e7 async-await: streaming hyper body example (#656)
Add reading the hyper body in the async/await example.
2018-09-21 19:58:46 -07:00
Toby Lawrence 1119d572ee io: ensure ReadHalf/WriteHalf do not return WouldBlock directly (#655)
* io: ensure ReadHalf/WriteHalf do not return WouldBlock directly

These facades were passing back WouldBlock when the internal BiLock
couldn't be acquired, which does not fit the intended behavior.

Signed-off-by: Toby Lawrence <[email protected]>

* io: pull from the local crate, not crates.io
2018-09-21 14:59:07 -04:00
Stjepan Glavina 20ca59114a threadpool: impl Drop for Queue (#649)
We need to drain the queue when dropping, or else those `Arc<Task>`s
will be leaked.

Fixes #542
2018-09-21 10:20:41 -07:00
Eliza Weisman 3dd95a9ff1 Add max line length to LinesCodec (#632)
## Motivation

Currently, there is a potential denial of service vulnerability in the
`lines` codec. Since there is no bound on the buffer that holds data
before it is split into a new line, an attacker could send an unbounded
amount of data without sending a `\n` character. 

## Solution

This branch adds a `new_with_max_length` constructor for `LinesCodec`
that configures a limit on the maximum number of bytes per line. When
the limit is reached, the the overly long line will be discarded (in 
`max_length`-sized increments until a newline character or the end of the
buffer is reached. It was also necessary to add some special-case logic
to avoid creating an empty line when the length limit is reached at the 
character immediately _before_ a `\n` character.

Additionally, this branch adds new tests for this function, including a
test for changing the line limit in-flight.

## Notes

This branch makes the following changes from my original PR with
this change (#590):

- The whole too-long line is discarded at once in the first call to `decode`
  that encounters it.
- Only one error is emitted per too-long line.
- Made all the changes requested by @carllerche in
  https://github.com/tokio-rs/tokio/pull/590#issuecomment-420735023

Fixes: #186 

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-20 17:08:00 -07:00
Sven Marnach be67eda117 fix deprecation warning in test for FutureExt::deadline() (#651)
* silence deprecation warnings for deadline in tests
* add new integration test for timeout
2018-09-20 15:26:56 -07:00
Alexander Polakov e267a1922d Reexport TaskExecutor from tokio_current_thread (#652) 2018-09-19 14:46:13 -07:00
Eliza Weisman 9b456f48d9 Set RUST_BACKTRACE=1 on AppVeyor (#650)
## Motivation

Currently, the `RUST_BACKTRACE` environment variable is set to `1` on
Travis CI builds:
https://github.com/tokio-rs/tokio/blob/0ca973a7ebc5b8a29beac1ccb6c73ef26ddcbf22/.travis.yml#L49
However, it's not set on AppVeyor. This can make debugging
Windows-specific CI failures challenging for developers on other
operating systems.

## Solution

This branch sets `RUST_BACKTRACE=1` on AppVeyor.

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-19 11:53:11 -07:00
Eliza Weisman 0ca973a7eb tokio: deprecate and replace runtime::threadpool_builder (#645)
* Deprecate and hide runtime::Builder::threadpool_builder

* Add functions to runtime::Builder wrapping threadpool builder functions

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-19 10:37:45 -04:00
RT df6acf0c2a tokio-timer: reset timeout after elapsed in stream (#648) 2018-09-19 09:47:39 -04:00
Eliza Weisman 98d23b8b29 Make tokio::run panic if called from inside tokio::run (#646)
This is implemented by creating an `Enter` instance from within `run`.

This patch also introduces `Enter::block_on`.

Fixes #504
2018-09-18 21:57:21 -07:00
Eliza Weisman 85f8522536 tokio-executor: hide deprecated tokio-threadpool reexports (#644)
Fixes: #643
Signed-off-by: Eliza Weisman <[email protected]>
2018-09-18 23:57:17 -04:00
Liran Ringel d275341fb2 Fix tokio-async-await tests compile errors (#630) 2018-09-18 13:49:08 -07:00
Nick Cameron d735e5d527 async-await: update deps in tokio-async-await (#639) 2018-09-18 10:08:35 -07:00
Carl Lerche 4019198706 Add some missing future::Executor implementations (#563)
This adds an implementation of future::Executor for
`executor::DefaultExecutor` and `runtime::current_thread::Handle`.
2018-09-17 22:23:48 -07:00
Ivan Petkov 24dc85dc5e ci: Run cargo test with the --no-fail-fast flag (#635)
Since the CI runs all tests for all tokio crates, it is possible that a
sporadic failure in one crate can mask failures/successes of other
crates' tests.

Using the `--no-fail-fast` flag instructs cargo to run *all* tests
before failing the build. This will allow checking to see if any
relevant test cases still pass even if an unrelated test has failed.
2018-09-15 00:41:41 +00:00
Ivan Petkov aaa5adb7fd Merge pull request #634 from vorner/import-signal-2
Import the `tokio-signal` source from its original repo

Original repository can be found at https://github.com/alexcrichton/tokio-signal
2018-09-14 22:58:19 +00:00
Michal 'vorner' Vaner 5f68b3aaa1 signal: Remove Apache license
Whole tokio is MIT only, unifying.
2018-09-14 23:28:56 +02:00
Michal 'vorner' Vaner 2f69acbe9f signal: Fix tests after importing & linking
* Don't use tokio-core any more for tests. That one brings tokio from
  crates.io instead of the current workspace and two versions of that
  don't want to cooperate.
* Guard unix-specific examples on windows.
* Leave CI setup to top-level directory.
2018-09-14 23:28:47 +02:00
Michal 'vorner' Vaner 7e12f5c39e signal: Link tokio-signal and tokio crates
References in the Cargo.toml, various links.
2018-09-14 23:28:33 +02:00
Michal 'vorner' Vaner 462882b356 Merge tokio with tokio-signal 2018-09-14 23:27:22 +02:00
Michal 'vorner' Vaner 35687f1d18 signal: Move to tokio-signal subdirectory
As a preparation to merge with tokio.
2018-09-14 23:25:57 +02:00
Michal 'vorner' Vaner e7dc3a1091 signal: Use signal-hook for registration of signals
This saves some code and gets rid of quite some amount of unsafe code.
2018-09-14 23:25:21 +02:00
Ivan Petkov b594e240f9 signal: Bump version to 0.2.5 2018-09-14 23:25:21 +02:00
Ivan Petkov 605708dca6 signal: Fix a possible starvation with concurrent Signal polls
* Originally reported in alexcrichton/tokio-process#42
* The root cause appears to be due to two different PollEvented
instances trying to consume readiness events from the same file
descriptor.
* Previously we would simply swallow any `AlreadyExists` errors when
attempting to register the pipe receiver with the event loop. I'm not
sure if this means the PollEvented wrapper wasn't fully registered to
receive events, or maybe there is a potential race condition with how
PollEvented consumes mio readiness events. Using a fresh/duplicate file
descriptor appears to mitigate the issue, however.
* I was also not able to reproduce the issue as an isolated test case so
there is no regression test available within this crate (but we can add
one in tokio-process)
2018-09-14 23:25:08 +02:00
Carl Lerche cc40a4e7f0 Revert "Add max line length to LinesCodec (#590)"
This reverts commit 4ae6c997ee.
2018-09-12 10:34:37 -07:00
Eliza Weisman 4ae6c997ee Add max line length to LinesCodec (#590)
* codec: add new constructor `with_max_length ` to `LinesCodec`
* codec: add security note to docs

Signed-off-by: Eliza Weisman <[email protected]>

* Fix Rust 1.25 compatibility

* codec: Fix incorrect line lengths in tests (and add assertions)

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix off-by-one error in lines codec

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix call to decode rather than decode_eof in test

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix incorrect LinesCodec::decode_max_line_length

This bug was introduced after the fix for the off-by-one error.
Fortunately, the doctests caught it.

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Minor style improvements

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Don't allow LinesCodec length limit to be set after construction

Signed-off-by: Eliza Weisman <[email protected]>

* codec: change LinesCodec to error and discard line when at max length

* codec: Fix build on Rust 1.25

The slice patterns syntax wasn't supported yet in that release.

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Add test for out-of-bounds index when peeking

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix out of bounds index

* codec: Fix incomplete comment

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Add test for line decoder buffer underrun
2018-09-12 13:02:57 -04:00
Stjepan Glavina 0f44adf5f6 reactor: use LocalKey::try_with in sharded RW lock (#628)
@jonhoo reported a panic in the call to `LocalKey::with`, which occurs
when the reactor is dropped in the middle of TLS teardown. This PR
changes the call to `LocalKey::try_with` and handles the case when the
thread-local value has already been destroyed.
2018-09-11 16:48:06 -04:00
Flux Xu 19d5565442 Expose thread_pool::SpawnHandle (#604) 2018-09-10 15:51:15 -07:00
Ivan Petkov 98e76d9bc6 signal: Bump version to to 0.2.4 2018-09-10 11:30:08 -07:00
Alan Somers b7f5bc95fe signal: Actually make unix::bsd public 2018-09-10 11:30:07 -07:00
Ivan Petkov 90ea2f6c5b signal: Bump version to 0.2.3 2018-09-10 11:30:07 -07:00
Ivan Petkov 837c3934d5 signal: Also cfg gate the entire unix::bsd module 2018-09-10 11:30:07 -07:00
Alan Somers bcd42d11d9 signal: Move SIGINFO to a BSD-specific submodule 2018-09-10 11:30:07 -07:00
Alan Somers 8ad66d296f signal: Add CHANGELOG entry for SIGINFO. 2018-09-10 11:30:07 -07:00
Alan Somers d9edc26e97 signal: export SIGINFO on supported platforms. 2018-09-10 11:30:07 -07:00
Alan Somers f0ac62151b signal: Update tokio dependency
tokio::runtime::current_thread was added in 0.1.6
2018-09-10 11:30:06 -07:00
Ivan Petkov 214722a296 signal: Update CHANGELOG 2018-09-10 11:30:06 -07:00
Niv Kaminer b8f8145b62 signal: cast SIGINFO to be the same type as sa_flags regardless of platform 2018-09-10 11:30:06 -07:00
Ivan Petkov 32d3e0e1f9 signal: Bump version to 0.2.2 2018-09-10 11:30:06 -07:00
Ivan Petkov 266919add6 signal: Refactor Signal tests
* Added timeouts to all tests that were missing them
 - any issue we have will likely result in deadlocks/starvation so its
   best if all tests quickly timeout rather than require getting killed
   or have the CI timeout itself
* Added a `support` module and put a bunch of helpers there to DRY the
tests
2018-09-10 11:30:06 -07:00
Ivan Petkov c9ffd98b1e signal: Fix a potential Signal starvation based on creation order
* As observed in alexcrichton/tokio-signal#38, Signal instances can starve based on the order
they are created in, and this ordering appears to be platform/OS
specific
* The crux of the issue is that we woud only *attempt* to broadcast any
pending signals if we successfully read out at least one byte from the
global pipe.
* For reasons unclear to me, the affected Signal instance would get
woken up after the signal handler writes to the global pipe, but it
would immediately hit a WouldBlock error and give up, bypassing the
broadcast attempt (even though the pending flag was correctly set).
 - Maybe this has to do with OS specifics with how the bytes are
   delivered (or not), or with some complex interaction with tokio and
   the pipe registration. It seems fishy since strace logs didn't show
   the signal handler pipe write fail either, but I'm all out of ideas
* The fix appears simple: unconditionally attempt to broadcast any
pending signals *any* time a Driver instance is woken up.
* Since we perform an atomic check for each pending signal, we know that
each (coalesced) signal broadcast will happen at most once. If we were
supuriously woken up and no signals were pending, then nothing will be
yielded to any pollers of Signal
* The down side is that since each Signal instance polls a Driver
instance, each poll to Signal will essentially perform N atomic
operations (N = number of signals we support) in an attempt to broadcast
any pending signals.
 - However, we can revisit optimizing this better in the future

Fixes alexcrichton/tokio-signal#38
2018-09-10 11:30:06 -07:00
Ivan Petkov 2d4bfa1485 signal: Fix starvation of signal streams on drop of another instance
* We introduce a new global structure which keeps track of how many
signal streams have been registered with a given event loop (the event
loop is identified by its OS file descriptor)
* We only attempt to deregister our global evented pipe from any event
loop if and only if we are the last signal that was registered with it
2018-09-10 11:30:05 -07:00
Ivan Petkov b6ecfa251c signal: Add (failing) test case which exibits starvation on drop
* Currently, whenever a new signal stream is created we attempt to
register a global pipe with the event loop to drive events.
* We also (correctly) swallow any descriptor-already-registered errors
since the same pipe is always used
* However, we currently *deregister* the same global pipe *any time* a
Signal stream is dropped.
* This means that if 2 or more of Signal instances exist simultaneously
(even if listening for different signals) and one of them is dropped,
the remainder will starve (until any new signal is created again).
2018-09-10 11:30:05 -07:00
Ivan Petkov 3a81d7746a signal: Split up all integration tests to run in their own process
* Cargo runs each integration-style-test in its own process. Since the
tests use global data structures specific to the process, we should run
them in an isolated manner to avoid having cross-test interactions
* Fixes alexcrichton/tokio-signal#39
2018-09-10 11:30:05 -07:00
Daniel Wagner-Hall 6e1a833825 signal: Update mio dependency to 0.6.14
This allows tokio-signal to build with `-Z minimal-versions` - see
https://github.com/rust-lang/cargo/issues/5657#issuecomment-401110172
for more details.

Earlier versions depend on log 0.3.1, which itself depends on libc
0.1, which doesn't build on any post-1.0 version of rust.
2018-09-10 11:30:05 -07:00
Niv Kaminer 3f80953dee signal: account for definition mismatch on aarch64 android 2018-09-10 11:30:05 -07:00
Michael Hadley 4374f5be70 signal: Fix typo in README.md 2018-09-10 11:30:05 -07:00
jjl b23ab94cd5 signal: Change reference from 'tokio-core' to 'tokio' in README 2018-09-10 11:30:04 -07:00
Markus Westerlind 40c77bd17e signal: Version 0.2 2018-09-10 11:30:04 -07:00
Markus Westerlind 31b51004f2 signal: Increase number of signals to 33 for the sake of FreeBSD
Fixes alexcrichton/tokio-signal#21
2018-09-10 11:30:04 -07:00
Markus Westerlind 9a4e4f2308 signal: Don't use the depreceated new method of Registration 2018-09-10 11:30:04 -07:00
Markus Westerlind 484fda7a23 signal: Add an appveyor build file
It is not possible to test much on windows but this will at least verify
that it can be built
2018-09-10 11:30:04 -07:00
Markus Westerlind 45ba6e2652 signal: Remove test that were accidentally included in the ctrl-c example 2018-09-10 11:30:04 -07:00
Markus Westerlind e73b8a0cc9 signal: refactor: Prefer the implicit handle passing used by tokio 2018-09-10 11:30:03 -07:00
Markus Westerlind 209232befd signal: Ensure that the driver dies once the signal does
`tokio::run` expects that all futures finish processing so we can't
leave `Driver` around forever or `tokio::run` would never return.
2018-09-10 11:30:03 -07:00
Markus Westerlind f759e4d70f signal: panic 2018-09-10 11:30:03 -07:00
Markus Westerlind 1fdff707b8 signal: test: Add a test for ctrl_c on unix 2018-09-10 11:30:03 -07:00
Markus Westerlind e97e8cb7fe signal: Update the windows implementation to work with tokio
BREAKING CHANGE

`ctrl_c` now takes a `tokio_reactor::Handle`
2018-09-10 11:30:03 -07:00
Markus Westerlind 2848df9b6c signal: Run rustfmt 0.4.2 2018-09-10 11:30:03 -07:00
Alex Crichton 7a24ed7509 signal: Bump to 0.1.5 2018-09-10 11:30:02 -07:00
Alex Crichton 9c9760cfbb signal: Fix a bug with most recent tokio-core release 2018-09-10 11:30:02 -07:00
Alex Crichton 5ecd929b1a signal: Bump to 0.1.4 2018-09-10 11:30:02 -07:00
Alex Crichton 8ddebf4309 signal: Fix compile on Android
Closes alexcrichton/tokio-signal#19
2018-09-10 11:30:02 -07:00
Alex Crichton 0df1882f21 signal: Bump to 0.1.3 2018-09-10 11:30:02 -07:00
Alex Crichton 4fa1b2b58c signal: Update to winapi 0.3 2018-09-10 11:30:02 -07:00
Alex Crichton a4895fe364 signal: Tweak travis config 2018-09-10 11:30:02 -07:00
Alex Crichton 7ab97f99c8 signal: Clarify wording of license information in README.
This text historically was copied verbatim from rust-lang/rust's own README [1]
with the intention of licensing projects the same as rustc's own license, namely
a dual MIT/Apache-2.0 license. The clause about "various BSD-like licenses"
isn't actually correct for almost all projects other than rust-lang/rust and
the wording around "both" was slightly ambiguous.

This commit updates the wording to match more precisely what's in the
standard library [2], namely clarifying that there aren't any BSD-like licenses
in this repository and that the source is licensable under either license, at
your own discretion.

[1]: https://github.com/rust-lang/rust/tree/f0fe716dbcbf2363ab8f929325d32a17e51039d0#license
[2]: https://github.com/rust-lang/rust/blob/f0fe716dbcbf2363ab8f929325d32a17e51039d0/src/libstd/lib.rs#L5-L9
2018-09-10 11:30:01 -07:00
Alex Crichton 9bf3228f73 signal: Add an example for waiting on two signals
Relies on `Stream::select` to merge streams.

Closes alexcrichton/tokio-signal#16
2018-09-10 11:30:01 -07:00
Raphael Nestler d41c60e21d signal: Fix typo in README 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 934c596133 signal: Ctrl+C example: quit after 10 signals. 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 72e2209bd8 signal: Ctrl+C example: more explanations 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 6a7092b9f7 signal: Ctrl+C example: Defer stream initialisation (and explain how/why) 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 1c893ef6d3 signal: undo nested example cargo project
.. after learning about `cargo run --example`
2018-09-10 11:30:00 -07:00
Jules Kerssemakers 20e7598e8d signal: Ctrl+C example: Don't forget proper attribution for original example 2018-09-10 11:30:00 -07:00
Jules Kerssemakers 3db92496f6 signal: Ctrl+C example: highlight power of Stream::for_each() 2018-09-10 11:30:00 -07:00
Jules Kerssemakers da47cfbd58 signal: Ctrl+C example: clarify control flow after receiving Ctrl+C: unreachable!() 2018-09-10 11:30:00 -07:00
Jules Kerssemakers f5eadc74f1 signal: Ctrl+C example: add explanatory comments 2018-09-10 11:30:00 -07:00
Jules Kerssemakers 175f9afea9 signal: ctrl+C example: Prompt user to do something.
So we don't stay at a blank terminal without any feedback after `cargo run`
2018-09-10 11:30:00 -07:00
Jules Kerssemakers 48eda3fe2f signal: Upgrade ctrl+c example into cargo run-able version with proper Cargo.toml 2018-09-10 11:29:59 -07:00
Alex Crichton 010c2223ca signal: Touch up the sighup-example slightly 2018-09-10 11:29:59 -07:00
Jules Kerssemakers 36b58d8fa8 signal: new example: SIGHUP, shows how to receive other signals than ctrl+C 2018-09-10 11:29:59 -07:00
Alex Crichton c601f68c9f signal: Add some examples to crate docs
Closes alexcrichton/tokio-signal#11
2018-09-10 11:29:59 -07:00
Alex Crichton 78ca103f3a signal: Update to tokio-io 2018-09-10 11:29:59 -07:00
Michal 'vorner' Vaner 7da00f3832 signal: Use IDs that don't run out
Replace the sequential counting (which might be exhausted) by an address
of an object (in a box, so it doesn't change). This is also a unique, so
it is acceptable ID.
2018-09-10 11:29:59 -07:00
Michal 'vorner' Vaner edba77e8df signal: A test running multiple event loops
Run multiple loops (both in parallel and sequentially) to make sure
broadcasting to multiple of them works and we work even after the
initial loop has gone away.
2018-09-10 11:29:58 -07:00
Michal 'vorner' Vaner cf1afd2d90 signal: Style: Replace tabs with spaces
Mixing tabs and spaces breaks indentation for people (and github) if
they use different tab width.
2018-09-10 11:29:58 -07:00
Alex Crichton 8e58a9d8d4 signal: Add badges/categories 2018-09-10 11:29:58 -07:00
Alex Crichton 4afef9391a signal: Bump to 0.1.2 2018-09-10 11:29:58 -07:00
Alex Crichton b6bacc1ca3 signal: Clarify a comment 2018-09-10 11:29:58 -07:00
Alex Crichton 955cd2836d signal: Clear out old Signal on drop 2018-09-10 11:29:58 -07:00
Alex Crichton 70e4ed67ad signal: Touch up more impls and comments 2018-09-10 11:29:58 -07:00
Alex Crichton 699b9ab89e signal: Handle a few more errors 2018-09-10 11:29:57 -07:00
Alex Crichton ba0921a01d signal: Various cleanups:
* Drop nix/lazy_static
* Use previously registered handlers
* Handle some more errors
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 367cb56e02 signal: The driver task
Add the driver task, connecting the signal handler wakeups to the
wakeups of of the streams.

It is a prototype-quality code, a lot of cleanups and similar is needed.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 04d949c380 signal: Provide the new kind of Signal stream
Which is just a wrapper around the futures::sync::mpsc. The sender is in
a global registry.

The part that connects the wakeups to the senders in the registry
doesn't yet exist.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner ed4359bb26 signal: Implement the wake-up part of the new signal handling
Register the signal handler that wakes up someone through a self-pipe.
That someone doesn't yet exist, though.

Some dependencies (nix, lazy_static) added to speed up the prototyping
process. They are likely to be dropped in some future commits.

Some features (eg. preserving the previous signal handlers) are still
missing.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 4142dc2fae signal: Use tokio-core from git
Just for now, as we need some yet unreleased features.
2018-09-10 11:29:57 -07:00
Alex Crichton 468b037e4e signal: Update docs urls and such 2018-09-10 11:29:56 -07:00
Alex Crichton b33ae3cdd6 signal: Remove deprecated API usage on Windows 2018-09-10 11:29:56 -07:00
Alex Crichton 4519ac8e17 signal: Remove use of deprecated APIs on Unix 2018-09-10 11:29:56 -07:00
Alex Crichton 92b93ee176 signal: Bump to 0.1.1 2018-09-10 11:29:56 -07:00
Alex Crichton 635149e3ab signal: Ignore errors in signal handler
Closes alexcrichton/tokio-signal#3
2018-09-10 11:29:56 -07:00
Alex Crichton e28c350e31 signal: Update travis token 2018-09-10 11:29:56 -07:00
Chris Emerson 1a122018a2 signal: Trivial typo fix. 2018-09-10 11:29:55 -07:00
Alex Crichton f3f8ee431e signal: Remove SIGKILL reexport 2018-09-10 11:29:55 -07:00
Alex Crichton 61c4047c6a signal: Add symbolic reexports for common signals
Means you don't have to import libc!

Closes alexcrichton/tokio-signal#1
2018-09-10 11:29:55 -07:00
Alex Crichton 3486a61a0f signal: Update deps to point to crates.io 2018-09-10 11:29:55 -07:00
Alex Crichton 8291c3d462 signal: Start adding windows support 2018-09-10 11:29:55 -07:00
Alex Crichton 1b6893b6f6 signal: Track tokio-core master 2018-09-10 11:29:55 -07:00
Alex Crichton eca7f0760f signal: Initial commit 2018-09-10 11:29:50 -07:00
Ben Boeckel 89d969d518 StreamExt: add a trait for additional Stream methods (#573)
Primarily, it offers a `timeout` method for streams.
2018-09-07 15:43:03 -07:00
Carl Lerche 16664189c1 async-await: move examples into dedicated crate (#608)
This works around a bug in the cargo renaming feature as well as allows
the use of `[patch]` in the `Cargo.toml`.
2018-09-06 13:36:59 -04:00
Carl Lerche 6828870608 async-await: bump version to v0.1.2 (#619) 2018-09-04 15:02:15 -07:00
Nimi Wariboko Jr 89d0cda2e2 async-await: use new PinMut/PinBox location (#613)
The types moved in `std`. This patch updates tokio-async-await to
import `PinMut` and `PinBox` from the new location.

Ref: rust-lang/rust#53227
2018-09-04 13:19:10 -07:00
Stjepan Glavina 8052a9b348 guide: fix a few typos (#612) 2018-09-03 10:18:53 -07:00
ksqsf a5ac6c8b72 Fix undesired multi-line error message (#605) 2018-08-31 09:53:41 -07:00
Léo Gaspard 3a59526523 Document that timeout-ed futures will be polled at least once (#603)
tokio-util: document behavior of `StreamExt::timeout` when timeout = 0
2018-08-31 09:26:41 -04:00
Zachary Stewart 322a94f72f Update documentation for AsyncRead and AsyncWrite (#596)
tokio-io: update documentation for AsyncRead and AsyncWrite
2018-08-31 09:00:32 -04:00
Eunchong Yu c03b23355b Fix minimum version to export tokio::codec module (#594)
tokio-async-await: fix minimum version to export tokio::codec module (#594)
2018-08-31 08:02:02 -04:00
Eliza Weisman bc91bc5022 Fix non-terminating loop in tokio_io::length_delimited::FramedWrite (#576)
* tokio-io: fix non-terminating loop in length_delimited::FramedWrite (#497)
2018-08-31 06:31:43 -04:00
Flux Xu a7b053372f Add ThreadPool::spawn_handle (#602)
## Motivation

`tokio_threadpool::ThreadPool::spawn` has no return value.

## Solution

Add `ThreadPool::spawn_handle` which calls
`futures::sync::oneshot::spawn` to return a future represents the return
value.
2018-08-30 16:53:05 -07:00
Eliza Weisman 673fdb5cb3 Refactor codec::length_delimited (#575)
This patch refactors `length_delimited` to be implemented as a `Codec` and
use the default `Framed` wrapper types.

The original implementation did not do this in order to support vectored writes in the
write half. However, this implementation would be more efficient with small frames anyway.

If vectored writes are to be explored in the future, then it should be done holistically.

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-30 14:50:32 -07:00
Carl Lerche 97618746de readme: fix section ordering (#600) 2018-08-30 14:46:40 -07:00
Carl Lerche d8f8b59df9 guide: add a testing section to the contributing guide (#598) 2018-08-30 12:26:24 -07:00
Jon Gjengset 0745a9b88a Use spawn_local to spawn from local Handles (#565)
Previously, every call to `current_thread::Handle::spawn` would go
through a `mpsc` channel. This is unnecessary when the `Handle` is still
on the same thread as the current thread executor. This patch fixes that
by storing the `ThreadId` of the executor when it is created, and then
comparing against that when `Handle::spawn` is called. If the call is
made from the same thread, `spawn_local` is used directly.

Fixes #562.
2018-08-30 11:24:59 -07:00
Eliza Weisman a7f5ba28ba Bump minimum supported version & document support policy (#599)
* Bump minimum supported version & document support policy

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-29 20:35:27 -04:00
Josef Brandl cc3b6af7a3 Fix tokio-uds version (#580) 2018-08-28 15:12:45 -07:00
Martin Chaine 07e30ae923 net: rework tokio_tcp and tokio_udp re-exports (#548)
This patch keeps the primary net types in `tokio::net` and moves
secondary types to a protocol specific submodules.

Primary types are the ones that users are most likely to name (`TcpStream`,
`TcpListener`, `UdpSocket`, ...)

Secondary types are the operation futures.
2018-08-28 14:13:06 -07:00
Jason Davies 69d90ac7ee Fix a few typos in timer docs. (#569) 2018-08-28 11:00:42 -07:00
Carl Lerche d16032cf06 async-await: misc fixes and typos (#585) 2018-08-27 15:16:32 -07:00
Carl Lerche b479ce78d3 add experimental async/await support. (#582)
This patch adds experimental async/await support to Tokio. It does this
by adding feature flags to existing libs only where necessary in order
to add nightly specific code (mostly `Unpin` implementations). It then
provides a new crate: `tokio-async-await` which is a shim layer on top
of `tokio`.

The `tokio-async-await` crate is expected to look exactly like `tokio`
does, but with async / await support. This strategy reduces the amount
of cfg guarding in the main libraries.

This patch also adds `tokio-channel`, which is copied from futures-rs
0.1 and adds the necessary `Unpin` implementations. In general, futures
0.1 is mostly unmaintained, so it will make sense for Tokio to take over
maintainership of key components regardless of async / await support.
2018-08-27 12:24:51 -07:00
Michal 'vorner' Vaner 6e45e0ac61 re-export tokio-current-thread::spawn (#579)
Re-export it inside the tokio::runtime::current_thread, as the original
place (tokio::executor::current_thread) is hidden from documentation and
users need some way to spawn non-Send futures.
2018-08-25 12:51:49 -07:00
Ben Boeckel 82c5baa09b Spelling fixes (#571)
* docs: fix spelling and whitespace errors
2018-08-25 15:26:41 -04:00
Carl Lerche 7dc6404726 draft initial CONTRIBUTING guide (#567)
This guide was adopted from the node.js project.
2018-08-24 13:03:34 -07:00
Eliza Weisman 2e88e29fe9 Move tokio_io::codec::length_delimited module to tokio::codec (#568)
* Deprecate tokio-io::length_delimited
* Move `length_delimited` into `tokio::codec`

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-24 15:54:42 -04:00
Carl Lerche 07203408de Bump version to v0.1.8 (#566)
This also bumps a number of sub crates:

* tokio-executor (0.1.3)
* tokio-io (0.1.8)
* tokio-reactor (0.1.4)
* tokio-threadpool (0.1.6)
* tokio-timer (0.2.6)
* tokio-udp (0.1.2)
2018-08-24 08:58:26 -07:00
Carl Lerche 8bf2e9aeb0 Introduce Timeout and deprecate Deadline. (#558)
This patch introduces `Timeout`. This new type allows setting a timeout
both using a duration and an instant. Given this overlap with
`Deadline`, `Deadline` is deprecated.

In addition to supporting future timeouts, the `Timeout` combinator is
able to provide timeout functionality to streams. It does this by
applying a duration based timeout to each item being yielded.

The main reason for introducing `Timeout` is that a deadline approach
does not work with streams. Since `Timeout` needed to be introduced
anyway, keeping `Deadline` around does not make sense.
2018-08-22 20:39:46 -07:00
Carl Lerche cf184eb326 timer: Reduce size of Delay struct (#554)
* Remove `counted` field on `timer::Entry`.

It turns out that a better indicator of whether or not the number of
active timeouts should be decremented is if the `Entry` has been
associated with a timer. In other words, if `Entry::inner` can be
upgraded, then the count should be decremented on drop.

* timer: Tweak link between `Delay` and the driver

This tweaks the struct layout / details regarding how a `Delay` instance
is linked to a driver (timer instance). Instead of lazily allocating the
`Entry` (node shared between `Delay` and the timer), `Entry` is
allocated immediately when `Delay` is created. This allows using the
entry store data used by `Delay`.

This is in anticipation of further timer improvements that would
otherwise require the size of `Delay` to grow further. Since an
allocation is already made, the idea is to shrink the size of the
`Delay` struct.
2018-08-21 21:48:40 -07:00
Carl Lerche d822b721b4 Add DelayQueue implementation to tokio-timer (#550)
This patch adds a `DelayQueue` to tokio_timer. The `DelayQueue` allows
inserting elements as well as specifying a time at which the element
should be returned to the user. This allows handling more complex
timeout situations.
2018-08-20 21:47:10 -07:00
Carl Lerche c66b56c3fb Implement Default for tokio_timer::Handle (#553)
This patch implements `Default` for `tokio_timer::Handle`. It returns a
`Handle` instance that is not bound to a specific timer. Instead, it
will use the timer for the current execution context. This is the same
strategy used by `tokio_reactor::Handle`.

Fixes #547
2018-08-20 13:01:39 -07:00
Carl Lerche 89639ec48b Bump tokio-uds version to v0.2.1 (#552)
Fixes #551
2018-08-19 21:23:49 -07:00
Martin Chaine 2b1b0ac858 Expose tokio_uds from the root crate (#526) 2018-08-15 21:26:10 -07:00
Carl Lerche 6b84c73f12 tokio::codec docs + additional exports (#546) 2018-08-15 21:25:25 -07:00
Jason Ish 767b370c21 Add tokio-tls echo example. (#541)
Based on the current example on the front page of tokio.rs.
2018-08-15 07:54:24 -04:00
Gary M. Josack 28010b5962 Update lines_encoder test to use LinesCodec (#544)
The `lines_encoder` test is a copy/paste of `bytes_encoder` and not
testing the LinesCodec encoding at all. This updates the test to do
simple validations of the LinesCodec encoding.
2018-08-15 07:51:42 -04:00
Roman 2e343f9e42 Reexport Encoder, Decoder, Framed* from tokio::codec (#499) 2018-08-14 11:18:54 -07:00
Mateusz Mikuła 31f71dedee Routine dependencies update (#533)
* Update dependencies

* Replace deprecated tempdir with tempfile
2018-08-10 12:37:45 -07:00
Stjepan Glavina 989262fe6e Enable sanitizer tests for tokio-threadpool (#537)
Closes #536.
2018-08-10 19:13:51 +02:00
Carl Lerche d91c775f36 Remove dead futures2 code. (#538)
The futures 0.2 crate is not intended for widespread usage. Also, the
futures team is exploring the compat shim route.

If futures 0.3 support is added to Tokio 0.1, then a different
integration route will be explored, making the current code unhelpful.
2018-08-09 21:56:53 -07:00
Stjepan Glavina 96b556fbff Steal multiple tasks from another worker at a time (#534)
* Steal multiple tasks from another worker at a time
* Better spinning and failing pop
* Update crossbeam-deque and simplify spinning
2018-08-09 12:14:13 -07:00
Stjepan Glavina fd36054ae4 Use a scalable RW lock in tokio-reactor (#517) 2018-08-09 11:23:45 -07:00
Carl Lerche 89d6bfc5cb Bump tokio-tls to v0.2.0 (#531)
This prepares the crate for release.
2018-08-08 10:17:28 -07:00
David Kellum decc83e959 Update to crossbeam-utils 0.5.0, fix imports (#519) 2018-08-08 08:57:25 -07:00
Sean McArthur afcfefd7e3 Move tokio-tls into workspace (#529) 2018-08-08 08:36:17 -07:00
Roman c89b0b4c8c Fix num CPUs in threadpool::builder::Builder::new (#530)
Closes #400
2018-08-08 08:33:01 -07:00
Stjepan Glavina 6b1e4ab0a3 Implement Error for a few error types (#511) 2018-08-07 19:45:58 -07:00
David Kellum 4153cc4076 Fix more rustdoc links (#518)
* Fix a cut-paste error with -reactor rustdoc links
* Fix more ::reactor rustdoc broken links
* Minor rustdoc typo
* Consistently reference std::io::{Read, Write} in rustdoc/links
2018-08-07 19:45:39 -07:00
Serho Liu 5304557d1d Fix tokio threadpool readme examples (#521) 2018-08-07 19:44:45 -07:00
Andrew Cann fdb2f61357 Udp socket readiness methods (#522) 2018-08-07 19:41:27 -07:00
Carl Lerche e964c4136c Bump subcrate versions (#524)
* tokio-current-thread 0.1.1
* tokio-executor 0.1.3
* tokio-fs 0.1.3
* tokio-reactor 0.1.3
* tokio-tcp 0.1.1
* tokio-timer 0.2.5
2018-08-06 20:36:50 -07:00
Brian Olsen 0490280d66 tokio-fs: Add async versions of most of std::fs (#494)
* create_dir
* create_dir_all
* hard_link
* read_dir
* read_link
* remove_dir
* remove_file
* rename
* set_permissions that works with path
* symlink_metadata
* symlink on unix
* symlink_dir on windows
* symlink_file on windows
2018-07-31 21:39:27 -07:00
Sam Rijs 0f76470172 detect and handle recursive calls to DefaultExecutor (#473) 2018-07-30 20:59:08 -07:00
Stjepan Glavina 9352249c3e Terminate backup threads when idle (#489) 2018-07-30 20:48:53 -07:00
Stjepan Glavina e5b2681513 Fix a race in thread wakeup (#507) 2018-07-30 20:46:46 -07:00
Stjepan Glavina 629c9f0698 Small fixes (#508)
* Make Shutdown public
* Remove unused import
* Fix documentation mistake
* Fix typo
2018-07-30 20:46:04 -07:00
Alan Somers 5d0d2a2e12 Ignore tokio-uds's test_socket_pair on FreeBSD. (#493)
It requires FreeBSD 12.0 or later.  Also, fix a spelling mistake in a
comment.
2018-07-24 14:00:01 -07:00
kohensu ad4693a18f Fix the doc of read_to_end method (#482) 2018-07-24 13:57:15 -07:00
Laurențiu Nicola c85bde3170 tokio: expose tokio_fs::metadata (#479) 2018-07-24 13:56:42 -07:00
Jon Gjengset 1e90e27720 Count in-transit spawned futures to current thread executor as pending (#478) 2018-07-24 13:49:01 -07:00
Carl Lerche f212a2ab9d Fix Weak tsan whitelist (#505) 2018-07-24 13:37:48 -07:00
Michal 'vorner' Vaner 84db325628 RunError and few more error types implements Error (#501)
This allows them to be used with things like `failure`.
2018-07-24 13:27:57 -07:00
Douman 365efec24a Add Interval::interval shortcut for a better usability (#492) 2018-07-23 23:08:49 -07:00
David Kellum 491f15827b General rustdoc improvements (#450)
* Normalize links to docs.rs/CRATE/M.N/...

docs.rs is smart enough to show docs for the latest M.N.P release when
M.N is used in the link. For example:

  https://docs.rs/mio/0.6/mio/struct.Poll.html

..will show mio 0.6.14 and later docs. While using the `M.N.*`
(ASTERISK) syntax also works, `M.N` is the more common usage, so
standarize a few existing links to that format.

* Fix missing or malformed rustdoc links

* executor lib rustdoc minor format change

* Promote tokio-threadpool crate level comments to rustdoc

* Replace hidden tokio::executor::thread_pool docs with deprecation note

* Fix typo/simplify util module rustdoc

* Reuse some tokio::executor::thread_pool rustdoc for the crate

Relates to #421
2018-07-22 13:35:30 -07:00
Stjepan Glavina c17ecb53e7 Pad fields to cacheline size to avoid false sharing (#475) 2018-07-16 14:22:48 -07:00
Jon Gjengset 6ba8e7621d Add free block_on_all in current thread Runtime (#477) 2018-07-11 15:32:58 -07:00
Laurențiu Nicola 39c95d6206 tokio-fs: Bump version to 0.1.2 (#469)
* Add a couple of missing full stops in the documentation
2018-07-11 15:09:37 -07:00
Sam Rijs 78b6bd4ca5 implement Send and Sync for DefaultExecutor (#472)
Fxes #376
2018-07-11 12:35:32 -07:00
Richard Dodd (dodj) b3ff9e315c Update lib.rs (#471)
Fix build failure on nightly (combination of warning for "cannot be resolved" and lint deny(warnings))
2018-07-11 12:30:51 -07:00
Stjepan Glavina 990186ec9d Optimize spinning in Worker::run (#470) 2018-07-11 12:30:26 -07:00
Stjepan Glavina 19da6ff59a New version of crossbeam-deque (#468) 2018-07-11 12:24:10 -07:00
Roman 36c817f0c3 Update rand dep from 0.4 to 0.5 (#458) 2018-07-11 12:14:40 -07:00
David Kellum 35123f7ae4 Additional details for tokio-fs rustdoc (#454) 2018-07-11 12:13:16 -07:00
João Oliveira 54b7c1b10d tokio-tcp: add tokio::net::TcpStream::try_clone (#448) 2018-07-11 11:54:08 -07:00
Patrick Barrett e6fc3d209d return NotReady when recv_from wouldblock in uds (#452) 2018-07-11 11:37:57 -07:00
Carl Lerche f98b81e527 Bump minimum supported Rust to 1.25. (#465)
Currenty, 1.27 is the latest released Rust version.
2018-07-06 14:14:05 -07:00
Stjepan Glavina dc7202cfa9 Replace XorShiftRng with a custom RNG (#466) 2018-07-06 13:33:52 -07:00
Carl Lerche f1a7caea3f Bump tokio-threadpool to v0.1.5 (#462) 2018-07-05 10:22:03 -07:00
Stjepan Glavina b019532bc2 Implement status() for DefaultExecutor (#463) 2018-07-05 10:19:49 -07:00
Stjepan Glavina 7fb579c667 Fix a race in thread wakeup (#459) 2018-07-03 16:28:50 -07:00
Stjepan Glavina dbefa67058 Make WorkerId public (#460) 2018-07-02 13:34:08 -07:00
Roman 24d99c029e Add a verbose error message for BlockingError (#451)
Add a verbose error message for EnterError while trying to run
tokio_threadpool::blocking on a current_thread::Runtime
2018-06-26 08:37:48 -07:00
Laurențiu Nicola 3fecd0154c Add an explicit wait for the test to finish (#445) 2018-06-22 14:07:02 -07:00
Laurențiu Nicola 0440343a11 tokio-fs: add changelog for 0.1.2 (#444) 2018-06-22 14:06:50 -07:00
Roman 3cf56b7bfa Fix unneeded mut and some deprecated api (#442) 2018-06-21 09:47:21 -07:00
Roman 7153d8d6ce Add a verbose error message for EnterError (#441)
Add a verbose error message for EnterError while trying to run an
executor while another executor is already running.

Fixes: #410
2018-06-21 09:46:45 -07:00
Laurențiu Nicola ecfe2f6a05 tokio-fs: add tokio_fs::File::seek (#434) 2018-06-21 09:43:35 -07:00
Laurențiu Nicola 5753553ba3 Move metadata to a submodule (#439) 2018-06-21 09:41:38 -07:00
Laurențiu Nicola 04a4bfd455 tokio-fs: add tokio_fs::metadata (#433) 2018-06-20 13:12:06 -07:00
Jake Goulding b2f77dcebe Add a dedicated Future for retrieving the metadata of a file (#385) 2018-06-18 16:00:43 -07:00
Carl Lerche 85cf47de86 Enable backtraces in CI & disable TSAN (#436)
This PR enables backtraces when running tests and disables tsan for the thread pool.

The thread sanitizer was generating too many false positives. Once #329 lands, then it can
be re-enabled.
2018-06-18 15:15:45 -07:00
Steven Fackler 45bcea6c4f Reexport tokio_uds::ConnectFuture (#430) 2018-06-18 13:26:06 -07:00
Carl Lerche 3fac7ce68c Add some thread pool docs (#421) 2018-06-15 15:20:25 -07:00
Marc-Antoine Perennou 71c8f561e3 runtime: add block_on_all (#398)
Signed-off-by: Marc-Antoine Perennou <[email protected]>
2018-06-14 22:13:39 -07:00
Sean McArthur 011ebf44eb Implement Executor for Box<E: Executor> (#420) 2018-06-14 16:28:23 -07:00
Carl Lerche c25ea78ec9 Bump version of a number of sub crates (#414)
This includes:

* tokio-codec (0.1.0)
* tokio-current-thread (0.1.0)
* tokio-fs (0.1.1)
* tokio-io (0.1.7)
* tokio-reactor (0.1.2)
* tokio-udp (0.1.1)
2018-06-13 10:24:56 -07:00
Carl Lerche 2e0cd292d2 Fix some broken doc links (#413) 2018-06-13 09:02:46 -07:00
Sylwek 4ebaf18c27 Typo (#415) 2018-06-13 09:02:34 -07:00
Carl Lerche ab07733d66 Deprecate executor re-exports (#412) 2018-06-12 14:41:12 -07:00
Mat Sadler d1f825ca13 Add OpenOptions to tokio-fs (#390)
Add an `OpenOptions` struct to `tokio-fs` that mirrors the one found in
`std`. Also provide a conversion from a `std` instance to a Tokio instance.
2018-06-12 10:47:24 -07:00
Laurențiu Nicola 4cf7d73b22 tokio-fs: add into_std (#403) 2018-06-12 10:40:43 -07:00
jpbriquet 2cd854c2c7 tokio-current-thread crate (#370)
Extract `tokio::executor::current_thread` to a tokio-current-thread
crate. Deprecated fns stay in the old location. The new crate only
contains thee most recent API.
2018-06-12 10:26:03 -07:00
Carl Lerche ba05c39d65 Fix a deadlock that can happen when shutting down (#409)
There is a deadlock that can occur when the concurrent runtime shuts
down. This patch adds a test and fix.

Fixes #401.
2018-06-12 09:41:18 -07:00
Alyssa Ross 64b8884911 Fix typo in comment (#402) 2018-06-11 15:26:55 -07:00
pravic d391e63418 Duplicated word in documentation. (#405) 2018-06-11 15:17:09 -07:00
Carl Lerche 8d8c895a1c Remove tokio-codec dependency from tokio (#397)
This will be added again later once types are re-exported.
2018-06-08 09:56:40 -07:00
630 changed files with 56111 additions and 31681 deletions
-19
View File
@@ -1,19 +0,0 @@
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --target %TARGET%
+42
View File
@@ -0,0 +1,42 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_PREEMPTIONS: 2
RUSTFLAGS: -Dwarnings
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y --profile minimal --default-toolchain stable
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
test_script:
- . $HOME/.cargo/env
- cargo test --all
- cargo doc --all --no-deps
# TODO: Re-enable
# i686_test_script:
# - . $HOME/.cargo/env
# - |
# cargo test --all --exclude tokio-tls --exclude tokio-macros --target i686-unknown-freebsd
+51
View File
@@ -0,0 +1,51 @@
<!--
Thank you for reporting an issue.
Please fill in as much of the template below as you're able.
-->
## Version
<!--
List the versions of all `tokio` crates you are using. The easiest way to get
this information is using `cargo-tree`.
`cargo install cargo-tree`
(see install here: https://github.com/sfackler/cargo-tree)
Then:
`cargo tree | grep tokio`
-->
## Platform
<!---
Output of `uname -a` (UNIX), or version and 32 or 64-bit (Windows)
-->
## Subcrates
<!--
If known, please specify the affected Tokio sub crates. Otherwise, delete this
section.
-->
## Description
<!--
Enter your issue details below this comment.
One way to structure the description:
<short summary of the bug>
I tried this code:
<code sample that causes the bug>
I expected to see this happen: <explanation>
Instead, this happened: <explanation>
-->
+23
View File
@@ -0,0 +1,23 @@
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.
Bug fixes and new features should include tests.
Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->
## Motivation
<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->
## Solution
<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
-95
View File
@@ -1,95 +0,0 @@
---
language: rust
sudo: false
cache:
- apt
- cargo
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
- rust: 1.21.0
- rust: stable
- rust: beta
- rust: nightly
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
script:
- |
set -e
if [[ "$TRAVIS_RUST_VERSION" == nightly ]]
then
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests
fi
- |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --target $TARGET
cargo check --tests --all --target $TARGET
else
cargo test --all
# Disable these tests for now as they are buggy
#
# cargo test --features unstable-futures
# cargo test --manifest-path tokio-threadpool/Cargo.toml --features unstable-futures
# cargo test --manifest-path tokio-reactor/Cargo.toml --features unstable-futures
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = linux
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
-48
View File
@@ -1,48 +0,0 @@
# 0.1.7 (June 6, 2018)
* Add `Runtime::block_on` for concurrent runtime (#391).
* Provide handle to `current_thread::Runtime` that allows spawning tasks from
other threads (#340).
* Provide `clock::now()`, a configurable source of time (#381).
# 0.1.6 (May 2, 2018)
* Add asynchronous filesystem APIs (#323).
* Add "current thread" runtime variant (#308).
* `CurrentThread`: Expose inner `Park` instance.
* Improve fairness of `CurrentThread` executor (#313).
# 0.1.5 (March 30, 2018)
* Provide timer API (#266)
# 0.1.4 (March 22, 2018)
* Fix build on FreeBSD (#218)
* Shutdown the Runtime when the handle is dropped (#214)
* Set Runtime thread name prefix for worker threads (#232)
* Add builder for Runtime (#234)
* Extract TCP and UDP types into separate crates (#224)
* Optionally support futures 0.2.
# 0.1.3 (March 09, 2018)
* Fix `CurrentThread::turn` to block on idle (#212).
# 0.1.2 (March 09, 2018)
* Introduce Tokio Runtime (#141)
* Provide `CurrentThread` for more flexible usage of current thread executor (#141).
* Add Lio for platforms that support it (#142).
* I/O resources now lazily bind to the reactor (#160).
* Extract Reactor to dedicated crate (#169)
* Add facade to sub crates and add prelude (#166).
* Switch TCP/UDP fns to poll_ -> Poll<...> style (#175)
# 0.1.1 (February 09, 2018)
* Doc fixes
# 0.1.0 (February 07, 2018)
* Initial crate released based on [RFC](https://github.com/tokio-rs/tokio-rfcs/pull/3).
+443
View File
@@ -0,0 +1,443 @@
# Contributing to Tokio
:balloon: Thanks for your help improving the project! We are so happy to have
you!
There are opportunities to contribute to Tokio at any level. It doesn't matter if
you are just getting started with Rust or are the most weathered expert, we can
use your help.
**No contribution is too small and all contributions are valued.**
This guide will help you get started. **Do not let this guide intimidate you**.
It should be considered a map to help you navigate the process.
The [dev channel][dev] is available for any concerns not covered in this guide, please join
us!
[dev]: https://discord.gg/6yGkFeN
## Conduct
The Tokio project adheres to the [Rust Code of Conduct][coc]. This describes
the _minimum_ behavior expected from all contributors.
[coc]: https://github.com/rust-lang/rust/blob/master/CODE_OF_CONDUCT.md
## Contributing in Issues
For any issue, there are fundamentally three ways an individual can contribute:
1. By opening the issue for discussion: For instance, if you believe that you
have uncovered a bug in Tokio, creating a new issue in the tokio-rs/tokio
issue tracker is the way to report it.
2. By helping to triage the issue: This can be done by providing
supporting details (a test case that demonstrates a bug), providing
suggestions on how to address the issue, or ensuring that the issue is tagged
correctly.
3. By helping to resolve the issue: Typically this is done either in the form of
demonstrating that the issue reported is not a problem after all, or more
often, by opening a Pull Request that changes some bit of something in
Tokio in a concrete and reviewable manner.
**Anybody can participate in any stage of contribution**. We urge you to
participate in the discussion around bugs and participate in reviewing PRs.
### Asking for General Help
If you have reviewed existing documentation and still have questions or are
having problems, you can open an issue asking for help.
In exchange for receiving help, we ask that you contribute back a documentation
PR that helps others avoid the problems that you encountered.
### Submitting a Bug Report
When opening a new issue in the Tokio issue tracker, users will be presented
with a [basic template][template] that should be filled in. If you believe that you have
uncovered a bug, please fill out this form, following the template to the best
of your ability. Do not worry if you cannot answer every detail, just fill in
what you can.
The two most important pieces of information we need in order to properly
evaluate the report is a description of the behavior you are seeing and a simple
test case we can use to recreate the problem on our own. If we cannot recreate
the issue, it becomes impossible for us to fix.
In order to rule out the possibility of bugs introduced by userland code, test
cases should be limited, as much as possible, to using only Tokio APIs.
See [How to create a Minimal, Complete, and Verifiable example][mcve].
[mcve]: https://stackoverflow.com/help/mcve
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Triaging a Bug Report
Once an issue has been opened, it is not uncommon for there to be discussion
around it. Some contributors may have differing opinions about the issue,
including whether the behavior being seen is a bug or a feature. This discussion
is part of the process and should be kept focused, helpful, and professional.
Short, clipped responses—that provide neither additional context nor supporting
detail—are not helpful or professional. To many, such responses are simply
annoying and unfriendly.
Contributors are encouraged to help one another make forward progress as much as
possible, empowering one another to solve issues collaboratively. If you choose
to comment on an issue that you feel either is not a problem that needs to be
fixed, or if you encounter information in an issue that you feel is incorrect,
explain why you feel that way with additional supporting context, and be willing
to be convinced that you may be wrong. By doing so, we can often reach the
correct outcome much faster.
### Resolving a Bug Report
In the majority of cases, issues are resolved by opening a Pull Request. The
process for opening and reviewing a Pull Request is similar to that of opening
and triaging issues, but carries with it a necessary review and approval
workflow that ensures that the proposed changes meet the minimal quality and
functional guidelines of the Tokio project.
## Pull Requests
Pull Requests are the way concrete changes are made to the code, documentation,
and dependencies in the Tokio repository.
Even tiny pull requests (e.g., one character pull request fixing a typo in API
documentation) are greatly appreciated. Before making a large change, it is
usually a good idea to first open an issue describing the change to solicit
feedback and guidance. This will increase the likelihood of the PR getting
merged.
### Tests
If the change being proposed alters code (as opposed to only documentation for
example), it is either adding new functionality to Tokio or it is fixing
existing, broken functionality. In both of these cases, the pull request should
include one or more tests to ensure that Tokio does not regress in the future.
There are two ways to write tests: integration tests and documentation tests
(Tokio avoids unit tests as much as possible).
#### Integration tests
Integration tests go in the same crate as the code they are testing. Each sub
crate should have a `dev-dependency` on `tokio` itself. This makes all Tokio
utilities available to use in tests, no matter the crate being tested.
The best strategy for writing a new integration test is to look at existing
integration tests in the crate and follow the style.
#### Documentation tests
Ideally, every API has at least one [documentation test] that demonstrates how to
use the API. Documentation tests are run with `cargo test --doc`. This ensures
that the example is correct and provides additional test coverage.
The trick to documentation tests is striking a balance between being succinct
for a reader to understand and actually testing the API.
Same as with integration tests, when writing a documentation test, the full
`tokio` crate is available. This is especially useful for getting access to the
runtime to run the example.
The documentation tests will be visible from both the crate specific
documentation **and** the `tokio` facade documentation via the re-export. The
example should be written from the point of view of a user that is using the
`tokio` crate. As such, the example should use the API via the facade and not by
directly referencing the crate.
The type level example for `tokio_timer::Timeout` provides a good example of a
documentation test:
```
/// // import the `timeout` function, usually this is done
/// // with `use tokio::prelude::*`
/// use tokio::prelude::FutureExt;
/// use futures::Stream;
/// use futures::sync::mpsc;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = mpsc::unbounded();
/// # tx.unbounded_send(()).unwrap();
/// # drop(tx);
///
/// let process = rx.for_each(|item| {
/// // do something with `item`
/// # drop(item);
/// # Ok(())
/// });
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// process.timeout(Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
Given that this is a *type* level documentation test and the primary way users
of `tokio` will create an instance of `Timeout` is by using
`FutureExt::timeout`, this is how the documentation test is structured.
Lines that start with `/// #` are removed when the documentation is generated.
They are only there to get the test to run. The `block_on_all` function is the
easiest way to execute a future from a test.
If this were a documentation test for the `Timeout::new` function, then the
example would explicitly use `Timeout::new`. For example:
```
/// use tokio::timer::Timeout;
/// use futures::Future;
/// use futures::sync::oneshot;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = oneshot::channel();
/// # tx.send(()).unwrap();
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// Timeout::new(rx, Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
### Commits
It is a recommended best practice to keep your changes as logically grouped as
possible within individual commits. There is no limit to the number of commits
any single Pull Request may have, and many contributors find it easier to review
changes that are split across multiple commits.
That said, if you have a number of commits that are "checkpoints" and don't
represent a single logical change, please squash those together.
Note that multiple commits often get squashed when they are landed (see the
notes about [commit squashing]).
#### Commit message guidelines
A good commit message should describe what changed and why.
1. The first line should:
* contain a short description of the change (preferably 50 characters or less,
and no more than 72 characters)
* be entirely in lowercase with the exception of proper nouns, acronyms, and
the words that refer to code, like function/variable names
* be prefixed with the name of the sub crate being changed (without the `tokio-`
prefix) and start with an imperative verb. If modifying `tokio` proper,
omit the crate prefix.
Examples:
* timer: introduce `Timeout` and deprecate `Deadline`
* export `Encoder`, `Decoder`, `Framed*` from tokio_codec
2. Keep the second line blank.
3. Wrap all other lines at 72 columns (except for long URLs).
4. If your patch fixes an open issue, you can add a reference to it at the end
of the log. Use the `Fixes: #` prefix and the issue number. For other
references use `Refs: #`. `Refs` may include multiple issues, separated by a
comma.
Examples:
- `Fixes: #1337`
- `Refs: #1234`
Sample complete commit message:
```txt
subcrate: explain the commit in one line
Body of commit message is a few lines of text, explaining things
in more detail, possibly giving some background about the issue
being fixed, etc.
The body of the commit message can be several paragraphs, and
please do proper word-wrap and keep columns shorter than about
72 characters or so. That way, `git log` will show things
nicely even when it is indented.
Fixes: #1337
Refs: #453, #154
```
### Opening the Pull Request
From within GitHub, opening a new Pull Request will present you with a
[template] that should be filled out. Please try to do your best at filling out
the details, but feel free to skip parts if you're not sure what to put.
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Discuss and update
You will probably get feedback or requests for changes to your Pull Request.
This is a big part of the submission process so don't be discouraged! Some
contributors may sign off on the Pull Request right away, others may have
more detailed comments or feedback. This is a necessary part of the process
in order to evaluate whether the changes are correct and necessary.
**Any community member can review a PR and you might get conflicting feedback**.
Keep an eye out for comments from code owners to provide guidance on conflicting
feedback.
**Once the PR is open, do not rebase the commits**. See [Commit Squashing] for
more details.
### Commit Squashing
In most cases, **do not squash commits that you add to your Pull Request during
the review process**. When the commits in your Pull Request land, they may be
squashed into one commit per logical change. Metadata will be added to the
commit message (including links to the Pull Request, links to relevant issues,
and the names of the reviewers). The commit history of your Pull Request,
however, will stay intact on the Pull Request page.
## Reviewing Pull Requests
**Any Tokio community member is welcome to review any pull request**.
All Tokio contributors who choose to review and provide feedback on Pull
Requests have a responsibility to both the project and the individual making the
contribution. Reviews and feedback must be helpful, insightful, and geared
towards improving the contribution as opposed to simply blocking it. If there
are reasons why you feel the PR should not land, explain what those are. Do not
expect to be able to block a Pull Request from advancing simply because you say
"No" without giving an explanation. Be open to having your mind changed. Be open
to working with the contributor to make the Pull Request better.
Reviews that are dismissive or disrespectful of the contributor or any other
reviewers are strictly counter to the Code of Conduct.
When reviewing a Pull Request, the primary goals are for the codebase to improve
and for the person submitting the request to succeed. **Even if a Pull Request
does not land, the submitters should come away from the experience feeling like
their effort was not wasted or unappreciated**. Every Pull Request from a new
contributor is an opportunity to grow the community.
### Review a bit at a time.
Do not overwhelm new contributors.
It is tempting to micro-optimize and make everything about relative performance,
perfect grammar, or exact style matches. Do not succumb to that temptation.
Focus first on the most significant aspects of the change:
1. Does this change make sense for Tokio?
2. Does this change make Tokio better, even if only incrementally?
3. Are there clear bugs or larger scale issues that need attending to?
4. Is the commit message readable and correct? If it contains a breaking change
is it clear enough?
Note that only **incremental** improvement is needed to land a PR. This means
that the PR does not need to be perfect, only better than the status quo. Follow
up PRs may be opened to continue iterating.
When changes are necessary, *request* them, do not *demand* them, and **do not
assume that the submitter already knows how to add a test or run a benchmark**.
Specific performance optimization techniques, coding styles and conventions
change over time. The first impression you give to a new contributor never does.
Nits (requests for small changes that are not essential) are fine, but try to
avoid stalling the Pull Request. Most nits can typically be fixed by the Tokio
Collaborator landing the Pull Request but they can also be an opportunity for
the contributor to learn a bit more about the project.
It is always good to clearly indicate nits when you comment: e.g.
`Nit: change foo() to bar(). But this is not blocking.`
If your comments were addressed but were not folded automatically after new
commits or if they proved to be mistaken, please, [hide them][hiding-a-comment]
with the appropriate reason to keep the conversation flow concise and relevant.
### Be aware of the person behind the code
Be aware that *how* you communicate requests and reviews in your feedback can
have a significant impact on the success of the Pull Request. Yes, we may land
a particular change that makes Tokio better, but the individual might just not
want to have anything to do with Tokio ever again. The goal is not just having
good code.
### Abandoned or Stalled Pull Requests
If a Pull Request appears to be abandoned or stalled, it is polite to first
check with the contributor to see if they intend to continue the work before
checking if they would mind if you took it over (especially if it just has nits
left). When doing so, it is courteous to give the original contributor credit
for the work they started (either by preserving their name and email address in
the commit log, or by using an `Author: ` meta-data tag in the commit.
_Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
## Releasing
Since the Tokio project consists of a number of crates, many of which depend on
each other, releasing new versions to crates.io can involve some complexities.
When releasing a new version of a crate, follow these steps:
1. **Ensure that the release crate has no path dependencies.** When the HEAD
version of a Tokio crate requires unreleased changes in another Tokio crate,
the crates.io dependency on the second crate will be replaced with a path
dependency. Crates with path dependencies cannot be published, so before
publishing the dependent crate, any path dependencies must also be published.
This should be done through a form of depth-first tree traversal:
1. Starting with the first path dependency in the crate to be released,
inspect the `Cargo.toml` for the dependency. If the dependency has any
path dependencies of its own, repeat this step with the first such
dependency.
2. Begin the release process for the path dependency.
3. Once the path dependency has been published to crates.io, update the
dependent crate to depend on the crates.io version.
4. When all path dependencies have been published, the dependent crate may
be published.
To verify that a crate is ready to publish, run:
```bash
bin/publish --dry-run <CRATE NAME> <CRATE VERSION>
```
2. **Update Cargo metadata.** After releasing any path dependencies, update the
`version` field in `Cargo.toml` to the new version, and the `documentation`
field to the docs.rs URL of the new version.
3. **Update other documentation links.** Update the `#![doc(html_root_url)]`
attribute in the crate's `lib.rs` and the "Documentation" link in the crate's
`README.md` to point to the docs.rs URL of the new version.
4. **Update the changelog for the crate.** Each crate in the Tokio repository
has its own `CHANGELOG.md` in that crate's subdirectory. Any changes to that
crate since the last release should be added to the changelog. Change
descriptions may be taken from the Git history, but should be edited to
ensure a consistent format, based on [Keep A Changelog][keep-a-changelog].
Other entries in that crate's changelog may also be used for reference.
5. **Perform a final audit for breaking changes.** Compare the HEAD version of
crate with the Git tag for the most recent release version. If there are any
breaking API changes, determine if those changes can be made without breaking
existing APIs. If so, resolve those issues. Otherwise, if it is necessary to
make a breaking release, update the version numbers to reflect this.
6. **Open a pull request with your changes.** Once that pull request has been
approved by a maintainer and the pull request has been merged, continue to
the next step.
7. **Release the crate.** Run the following command:
```bash
bin/publish <NAME OF CRATE> <VERSION>
```
Your editor and prompt you to edit a message for the tag. Copy the changelog
entry for that release version into your editor and close the window.
[keep-a-changelog]: https://github.com/olivierlacan/keep-a-changelog/blob/master/CHANGELOG.md
+10 -66
View File
@@ -1,70 +1,14 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.7"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio/0.1"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio-codec",
"tokio-executor",
"tokio-fs",
"tokio-io",
"tokio-reactor",
"tokio-threadpool",
"tokio-timer",
"tokio-tcp",
"tokio-udp",
"tokio-uds",
"tokio",
"tokio-macros",
"tokio-test",
"tokio-tls",
"tokio-util",
# Internal
"examples",
"tests-build",
"tests-integration",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
tokio-codec = { version = "0.1.0", path = "tokio-codec" }
tokio-io = { version = "0.1.6", path = "tokio-io" }
tokio-executor = { version = "0.1.2", path = "tokio-executor" }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor" }
tokio-threadpool = { version = "0.1.4", path = "tokio-threadpool" }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp" }
tokio-udp = { version = "0.1.0", path = "tokio-udp" }
tokio-timer = { version = "0.2.4", path = "tokio-timer" }
tokio-fs = { version = "0.1.0", path = "tokio-fs" }
futures = "0.1.20"
# Needed until `reactor` is removed from `tokio`.
mio = "0.6.14"
[dev-dependencies]
bytes = "0.4"
env_logger = { version = "0.4", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
+78 -79
View File
@@ -14,29 +14,23 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Gitter chat][gitter-badge]][gitter-url]
[![Build Status][azure-badge]][azure-url]
[![Discord chat][discord-badge]][discord-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[mit-url]: LICENSE
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[discord-badge]: https://img.shields.io/discord/500028886025895936.svg?logo=discord&style=flat-square
[discord-url]: https://discord.gg/6yGkFeN
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[Guides](https://tokio.rs/docs/) |
[API Docs](https://docs.rs/tokio/latest/tokio) |
[Roadmap](https://github.com/tokio-rs/tokio/blob/master/ROADMAP.md) |
[Chat](https://discord.gg/6yGkFeN)
## Overview
@@ -45,102 +39,107 @@ asynchronous applications with the Rust programming language. At a high
level, it provides a few major components:
* A multithreaded, work-stealing based task [scheduler].
* A [reactor] backed by the operating system's event queue (epoll, kqueue,
* A reactor backed by the operating system's event queue (epoll, kqueue,
IOCP, etc...).
* Asynchronous [TCP and UDP][net] sockets.
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/latest/tokio/net/index.html
[scheduler]: https://docs.rs/tokio/latest/tokio/runtime/index.html
## Example
A basic TCP echo server with Tokio:
```rust
extern crate tokio;
use tokio::prelude::*;
use tokio::io::copy;
```rust,no_run
use tokio::net::TcpListener;
use tokio::prelude::*;
fn main() {
// Bind the server's socket.
let addr = "127.0.0.1:12345".parse().unwrap();
let listener = TcpListener::bind(&addr)
.expect("unable to bind TCP listener");
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut listener = TcpListener::bind("127.0.0.1:8080").await?;
// Pull out a stream of sockets for incoming connections
let server = listener.incoming()
.map_err(|e| eprintln!("accept failed = {:?}", e))
.for_each(|sock| {
// Split up the reading and writing parts of the
// socket.
let (reader, writer) = sock.split();
loop {
let (mut socket, _) = listener.accept().await?;
// A future that echos the data and returns how
// many bytes were copied...
let bytes_copied = copy(reader, writer);
tokio::spawn(async move {
let mut buf = [0; 1024];
// ... after which we'll print what happened.
let handle_conn = bytes_copied.map(|amt| {
println!("wrote {:?} bytes", amt)
}).map_err(|err| {
eprintln!("IO error {:?}", err)
});
// In a loop, read data from the socket and write the data back.
loop {
let n = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => return,
Ok(n) => n,
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
return;
}
};
// Spawn the future as a concurrent task.
tokio::spawn(handle_conn)
// Write the data back
if let Err(e) = socket.write_all(&buf[0..n]).await {
eprintln!("failed to write to socket; err = {:?}", e);
return;
}
}
});
// Start the Tokio runtime
tokio::run(server);
}
}
```
More examples can be found [here](examples).
More examples can be found [here](examples). Note that the `master` branch
is currently being updated to use `async` / `await`. The examples are
not fully ported. Examples for stable Tokio can be found
[here](https://github.com/tokio-rs/tokio/tree/v0.1.x/tokio/examples).
## Project layout
The `tokio` crate, found at the root, is primarily intended for use by
application developers. Library authors should depend on the sub crates, which
have greater guarantees of stability.
## Getting Help
The crates included as part of Tokio are:
First, see if the answer to your question can be found in the [Guides] or the
[API documentation]. If the answer is not there, there is an active community in
the [Tokio Discord server][chat]. We would be happy to try to answer your
question. Last, if that doesn't work, try opening an [issue] with the question.
* [`tokio-executor`]: Task execution related traits and utilities.
[Guides]: https://tokio.rs/docs/
[API documentation]: https://docs.rs/tokio/latest/tokio
[chat]: https://discord.gg/6yGkFeN
[issue]: https://github.com/tokio-rs/tokio/issues/new
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
## Contributing
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
:balloon: Thanks for your help improving the project! We are so happy to have
you! We have a [contributing guide][guide] to help you get involved in the Tokio
project.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
[guide]: CONTRIBUTING.md
* [`tokio-tcp`]: TCP bindings for use with `tokio-io` and `tokio-reactor`.
## Related Projects
* [`tokio-threadpool`]: Schedules the execution of futures across a pool of
threads.
In addition to the crates in this repository, the Tokio project also maintains
several other libraries, including:
* [ `tokio-timer`]: Time related APIs.
* [`tracing`] (formerly `tokio-trace`): A framework for application-level
tracing and async-aware diagnostics.
* [`tokio-udp`]: UDP bindings for use with `tokio-io` and `tokio-reactor`.
* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers
`tokio`.
* [`tokio-uds`]: Unix Domain Socket bindings for use with `tokio-io` and
`tokio-reactor`.
* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-io`]: tokio-io
[`tokio-reactor`]: tokio-reactor
[`tokio-tcp`]: tokio-tcp
[`tokio-threadpool`]: tokio-threadpool
[`tokio-timer`]: tokio-timer
[`tokio-udp`]: tokio-udp
[`tokio-uds`]: tokio-uds
[`tracing`]: https://github.com/tokio-rs/tracing
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Supported Rust Versions
Tokio is built against the latest stable, nightly, and beta Rust releases. The
minimum version supported is the stable release from three months before the
current stable release version. For example, if the latest stable Rust is 1.29,
the minimum version supported is 1.26. The current Tokio version is not
guaranteed to build on Rust versions earlier than the minimum supported version.
## License
+67
View File
@@ -0,0 +1,67 @@
# Tokio Roadmap
## A Roadmap to 1.0
The question of "why not 1.0?" has come up a few times. After all, Tokio 0.1 has
been stable for three years. The short answer: because it isn't time. There is
nobody who would rather ship a Tokio 1.0 than us. It also isn't something to rush.
After all, `async / await` only landed in the stable Rust channel weeks ago.
There has been no significant production validation yet, except maybe fuchsia
and that seems like a fairly specialized use case. This release of Tokio
includes significant new code and new strategies with feature flags. Also, there
are still big open questions, such as the [proposed changes][pr-1744] to
`AsyncRead` and `AsyncWrite`.
Tokio 1.0 will be released as soon as the APIs are proven to handle real-world
production cases.
### Tokio 1.0 in Q3 2020 with LTS support
The Tokio 1.0 release will be **no later** than Q3 2020. It will also come with
"long-term support" guarantees:
* A minimum of 5 years of maintenance.
* A minimum of 3 years before a hypothetical 2.0 release.
When Tokio 1.0 is released in Q3 2020, on-going support, security fixes, and
critical bug fixes are guaranteed until **at least** Q3 2025. Tokio 2.0 will not
be released until **at least** Q3 2023 (though, ideally there will never been a
Tokio 2.0 release).
### How to get there
While Tokio 0.1 probably should have been a 1.0, Tokio 0.2 will be a **true**
0.2 release. There will breaking change releases every 2 ~ 3 months until 1.0.
These changes will be **much** smaller than going from 0.1 -> 0.2. It is
expected that the 1.0 release will look a lot like 0.2.
### What is expected to change
The biggest change will be the `AsyncRead` and `AsyncWrite` traits. Based on
experience gained over the past 3 years, there are a couple of issues to
address:
* Be able to **safely** use uninitialized memory as a read buffer.
* Practical read vectored and write vectored APIs.
There are a few strategies to solve these problems. These strategies need to be
investigated and the solution validated. You can see [this comment][pr-1744-comment] for a
detailed statement of the problem.
The other major change, which has been in the works for a while, is updating
Mio. Mio 0.6 was first released almost 4 years ago and has not had a breaking
change since. Mio 0.7 has been in the works for a while. It includes a full
rewrite of the windows support as well as a refined API. More will be written
about this shortly.
Finally, now that the API is starting to stabilize, effort will be put into
documentation. Tokio 0.2 is being released before updating the website and many
of the old content will no longer be relevant. In the coming weeks, expect to
see updates there.
So, we have our work cut out for us. We hope you enjoy this 0.2 release and are
looking forward to your feedback and help.
[pr-1744]: https://github.com/tokio-rs/tokio/pull/1744
[pr-1744-comment]: https://github.com/tokio-rs/tokio/pull/1744#issuecomment-553575438
+108
View File
@@ -0,0 +1,108 @@
trigger: ["master"]
pr: ["master"]
variables:
RUSTFLAGS: -Dwarnings
nightly: nightly-2019-11-16
jobs:
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
rust: stable
displayName: Test tokio
cross: true
crates:
- tokio
- tests-integration
# Test sub crates
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
rust: stable
crates:
- tokio-macros
- tokio-test
- tokio-tls
- tokio-util
- examples
# Run tests from `tests-build`. This requires a different process
- template: ci/azure-test-build.yml
parameters:
name: test_build
displayName: Test build permutations
rust: stable
# Run loom tests
- template: ci/azure-loom.yml
parameters:
name: loom
rust: stable
crates:
- tokio
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross
rust: stable
# Check each feature works properly
- template: ci/azure-check-features.yml
parameters:
rust: $(nightly)
name: check_features
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust: 1.39.0
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
rust: stable
name: rustfmt
# Apply clippy lints to all crates
- template: ci/azure-clippy.yml
parameters:
rust: stable
name: clippy
# Check doc generation
- template: ci/azure-check-docs.yml
parameters:
rust: $(nightly)
name: docs
# - template: ci/azure-tsan.yml
# parameters:
# name: tsan
# rust: stable
- template: ci/azure-deploy-docs.yml
parameters:
rust: stable
dependsOn:
- rustfmt
- clippy
- test_tokio
- test_linux
- test_build
- loom
- cross
- minrust
- check_features
# - tsan
-117
View File
@@ -1,117 +0,0 @@
#![feature(test)]
#![deny(warnings)]
extern crate test;
#[macro_use]
extern crate futures;
extern crate tokio;
use std::io;
use std::net::SocketAddr;
use std::thread;
use futures::sync::oneshot;
use futures::sync::mpsc;
use futures::{Future, Poll, Sink, Stream};
use test::Bencher;
use tokio::net::UdpSocket;
/// UDP echo server
struct EchoServer {
socket: UdpSocket,
buf: Vec<u8>,
to_send: Option<(usize, SocketAddr)>,
}
impl EchoServer {
fn new(s: UdpSocket) -> Self {
EchoServer {
socket: s,
to_send: None,
buf: vec![0u8; 1600],
}
}
}
impl Future for EchoServer {
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<(), io::Error> {
loop {
if let Some(&(size, peer)) = self.to_send.as_ref() {
try_ready!(self.socket.poll_send_to(&self.buf[..size], &peer));
self.to_send = None;
}
self.to_send = Some(try_ready!(self.socket.poll_recv_from(&mut self.buf)));
}
}
}
#[bench]
fn udp_echo_latency(b: &mut Bencher) {
let any_addr = "127.0.0.1:0".to_string();
let any_addr = any_addr.parse::<SocketAddr>().unwrap();
let (stop_c, stop_p) = oneshot::channel::<()>();
let (tx, rx) = oneshot::channel();
let child = thread::spawn(move || {
let socket = tokio::net::UdpSocket::bind(&any_addr).unwrap();
tx.send(socket.local_addr().unwrap()).unwrap();
let server = EchoServer::new(socket);
let server = server.select(stop_p.map_err(|_| panic!()));
let server = server.map_err(|_| ());
server.wait().unwrap();
});
let client = std::net::UdpSocket::bind(&any_addr).unwrap();
let server_addr = rx.wait().unwrap();
let mut buf = [0u8; 1000];
// warmup phase; for some reason initial couple of
// runs are much slower
//
// TODO: Describe the exact reasons; caching? branch predictor? lazy closures?
for _ in 0..8 {
client.send_to(&buf, &server_addr).unwrap();
let _ = client.recv_from(&mut buf).unwrap();
}
b.iter(|| {
client.send_to(&buf, &server_addr).unwrap();
let _ = client.recv_from(&mut buf).unwrap();
});
stop_c.send(()).unwrap();
child.join().unwrap();
}
#[bench]
fn futures_channel_latency(b: &mut Bencher) {
let (mut in_tx, in_rx) = mpsc::channel(32);
let (out_tx, out_rx) = mpsc::channel::<_>(32);
let child = thread::spawn(|| out_tx.send_all(in_rx.then(|r| r.unwrap())).wait());
let mut rx_iter = out_rx.wait();
// warmup phase; for some reason initial couple of runs are much slower
//
// TODO: Describe the exact reasons; caching? branch predictor? lazy closures?
for _ in 0..8 {
in_tx.start_send(Ok(1usize)).unwrap();
let _ = rx_iter.next();
}
b.iter(|| {
in_tx.start_send(Ok(1usize)).unwrap();
let _ = rx_iter.next();
});
drop(in_tx);
child.join().unwrap().unwrap();
}
-58
View File
@@ -1,58 +0,0 @@
// Measure cost of different operations
// to get a sense of performance tradeoffs
#![feature(test)]
#![deny(warnings)]
extern crate test;
extern crate mio;
use test::Bencher;
use mio::tcp::TcpListener;
use mio::{Token, Ready, PollOpt};
#[bench]
fn mio_register_deregister(b: &mut Bencher) {
let addr = "127.0.0.1:0".parse().unwrap();
// Setup the server socket
let sock = TcpListener::bind(&addr).unwrap();
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
b.iter(|| {
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.deregister(&sock).unwrap();
});
}
#[bench]
fn mio_reregister(b: &mut Bencher) {
let addr = "127.0.0.1:0".parse().unwrap();
// Setup the server socket
let sock = TcpListener::bind(&addr).unwrap();
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
b.iter(|| {
poll.reregister(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
});
poll.deregister(&sock).unwrap();
}
#[bench]
fn mio_poll(b: &mut Bencher) {
let poll = mio::Poll::new().unwrap();
let timeout = std::time::Duration::new(0, 0);
let mut events = mio::Events::with_capacity(1024);
b.iter(|| {
poll.poll(&mut events, Some(timeout)).unwrap();
});
}
-249
View File
@@ -1,249 +0,0 @@
#![feature(test)]
#![deny(warnings)]
extern crate futures;
extern crate tokio;
#[macro_use]
extern crate tokio_io;
pub extern crate test;
mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::executor::current_thread;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
pub use std::thread;
pub use std::time::Duration;
pub use std::io::{self, Read, Write};
}
mod connect_churn {
use ::prelude::*;
const NUM: usize = 300;
const CONCURRENT: usize = 8;
#[bench]
fn one_thread(b: &mut Bencher) {
let addr = "127.0.0.1:0".parse().unwrap();
b.iter(move || {
let listener = TcpListener::bind(&addr).unwrap();
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
let connects = stream::iter_result((0..NUM).map(|_| {
Ok(TcpStream::connect(&addr)
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
let connects_concurrent = connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()));
serve_incomings.select(connects_concurrent)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
});
}
fn n_workers(n: usize, b: &mut Bencher) {
let (shutdown_tx, shutdown_rx) = sync::oneshot::channel();
let (addr_tx, addr_rx) = sync::oneshot::channel();
// Spawn reactor thread
let server_thread = thread::spawn(move || {
// Bind the TCP listener
let listener = TcpListener::bind(
&"127.0.0.1:0".parse().unwrap()).unwrap();
// Get the address being listened on.
let addr = listener.local_addr().unwrap();
// Send the remote & address back to the main thread
addr_tx.send(addr).unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
// Run server
serve_incomings.select(shutdown_rx)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
});
// Get the bind addr of the server
let addr = addr_rx.wait().unwrap();
b.iter(move || {
use std::sync::{Barrier, Arc};
// Create a barrier to coordinate threads
let barrier = Arc::new(Barrier::new(n + 1));
// Spawn worker threads
let threads: Vec<_> = (0..n).map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
barrier.wait();
connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(())).wait().unwrap();
})
}).collect();
barrier.wait();
for th in threads {
th.join().unwrap();
}
});
// Shutdown the server
shutdown_tx.send(()).unwrap();
server_thread.join().unwrap();
}
#[bench]
fn two_threads(b: &mut Bencher) {
n_workers(1, b);
}
#[bench]
fn multi_threads(b: &mut Bencher) {
n_workers(4, b);
}
}
mod transfer {
use ::prelude::*;
use std::{cmp, mem};
const MB: usize = 3 * 1024 * 1024;
struct Drain {
sock: TcpStream,
chunk: usize,
}
impl Future for Drain {
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<(), io::Error> {
let mut buf: [u8; 1024] = unsafe { mem::uninitialized() };
loop {
match try_nb!(self.sock.read(&mut buf[..self.chunk])) {
0 => return Ok(Async::Ready(())),
_ => {}
}
}
}
}
struct Transfer {
sock: TcpStream,
rem: usize,
chunk: usize,
}
impl Future for Transfer {
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<(), io::Error> {
while self.rem > 0 {
let len = cmp::min(self.rem, self.chunk);
let buf = &DATA[..len];
let n = try_nb!(self.sock.write(&buf));
self.rem -= n;
}
Ok(Async::Ready(()))
}
}
static DATA: [u8; 1024] = [0; 1024];
fn one_thread(b: &mut Bencher, read_size: usize, write_size: usize) {
let addr = "127.0.0.1:0".parse().unwrap();
b.iter(move || {
let listener = TcpListener::bind(&addr).unwrap();
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts 1 connection, Drain it and drops
let server = listener.incoming()
.into_future() // take the first connection
.map_err(|(e, _other_incomings)| e)
.map(|(connection, _other_incomings)| connection.unwrap())
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
let drain = Drain {
sock: sock,
chunk: read_size,
};
drain.map(|_| ()).map_err(|e| panic!("server error: {:?}", e))
})
.map_err(|e| panic!("server err: {:?}", e));
let client = TcpStream::connect(&addr)
.and_then(move |sock| {
Transfer {
sock: sock,
rem: MB,
chunk: write_size,
}
})
.map_err(|e| panic!("client err: {:?}", e));
server.join(client).wait().unwrap();
});
}
mod small_chunks {
use ::prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
super::one_thread(b, 32, 32);
}
}
mod big_chunks {
use ::prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
super::one_thread(b, 1_024, 1_024);
}
}
}
Executable
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
set -e
USAGE="Publish a new release of a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-v, --verbose Use verbose Cargo output
-d, --dry-run Perform a dry run (do not publish or tag the release)
-h, --help Show this help text and exit"
DRY_RUN=""
VERBOSE=""
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
verify() {
status "Verifying" "if $CRATE v$VERSION can be released"
ACTUAL=$(cargo pkgid | sed -n 's/.*#\(.*\)/\1/p')
if [ "$ACTUAL" != "$VERSION" ]; then
err "expected to release version $VERSION, but Cargo.toml contained $ACTUAL"
exit 1
fi
if git tag -l | grep -Fxq "$TAG" ; then
err "git tag \`$TAG\` already exists"
exit 1
fi
PATH_DEPS=$(grep -F "path = \"" Cargo.toml | sed -e 's/^/ /')
if [ -n "$PATH_DEPS" ]; then
err "crate \`$CRATE\` contained path dependencies:\n$PATH_DEPS"
echo "path dependencies must be removed prior to release"
exit 1
fi
}
release() {
status "Releasing" "$CRATE v$VERSION"
cargo package $VERBOSE
cargo publish $VERBOSE $DRY_RUN
status "Tagging" "$TAG"
if [ -n "$DRY_RUN" ]; then
echo "# git tag $TAG && git push --tags"
else
git tag "$TAG" && git push --tags
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-v|--verbose)
VERBOSE="--verbose"
set +x
shift
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
(cd "$CRATE" && verify && release )
else
err "no such crate \"$CRATE\""
exit 1
fi
Executable
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
set -e
USAGE="Update links to docs.rs in a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-d, --dry-run Perform a dry run (do not modify any file)
-h, --help Show this help text and exit"
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
c1grep() { grep "$@" || test $? = 1; }
update_versions_in_doc() {
# Print what is being/would be done
if [ -n "$DRY_RUN" ]; then
local MSG="Would change:"
else
local MSG="Updating:"
fi
git grep -lr "docs.rs/$CRATE/" \
| xargs sed --quiet \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|gp" \
| sed -e "s/^/$MSG /"
# Apply changes if not in dry run
if [ -z "$DRY_RUN" ]; then
git grep -lr "docs.rs/$CRATE/" \
| xargs sed -i \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|g"
fi
}
update() {
update_versions_in_doc
}
show_outdated() {
OUTDATED=$(git grep -rn "docs.rs/$CRATE/" \
| c1grep -v "$VERSION" \
| sed -e 's/^/ - /')
if [[ -n "$OUTDATED" ]]; then
echo "Found the following links to docs.rs with an outdated version:"
echo "$OUTDATED"
echo
else
echo "Nothing to do."
exit 1
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
# Does not cd in order to update everywhere
show_outdated && update
else
err "no such crate \"$CRATE\""
exit 1
fi
+29
View File
@@ -0,0 +1,29 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+15
View File
@@ -0,0 +1,15 @@
jobs:
# Check docs
- job: ${{ parameters.name }}
displayName: Check docs
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
RUSTDOCFLAGS="--cfg docsrs" cargo doc --lib --no-deps --all-features
displayName: Check docs
+32
View File
@@ -0,0 +1,32 @@
jobs:
- job: ${{ parameters.name }}
displayName: Check features
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo install cargo-hack
displayName: Install cargo-hack
# Check each feature works properly
# * --each-feature
# run for each feature which includes --no-default-features and default features of package
# * -Z avoid-dev-deps
# build without dev-dependencies to avoid https://github.com/rust-lang/cargo/issues/4866
# tracking-issue: https://github.com/rust-lang/cargo/issues/5133
- script: cargo hack check --all --each-feature -Z avoid-dev-deps
displayName: cargo hack check --all --each-feature
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+16
View File
@@ -0,0 +1,16 @@
jobs:
- job: ${{ parameters.name }}
displayName: Clippy
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add clippy
cargo clippy --version
displayName: Install clippy
- script: |
cargo clippy --all --all-features -- -A clippy::mutex-atomic
displayName: cargo clippy --all
+44
View File
@@ -0,0 +1,44 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
i686:
vmImage: ubuntu-16.04
target: i686-unknown-linux-gnu
powerpc:
vmImage: ubuntu-16.04
target: powerpc-unknown-linux-gnu
powerpc64:
vmImage: ubuntu-16.04
target: powerpc64-unknown-linux-gnu
mips:
vmImage: ubuntu-16.04
target: mips-unknown-linux-gnu
arm:
vmImage: ubuntu-16.04
target: arm-linux-androideabi
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: sudo apt-get update
displayName: apt-get update
- script: sudo apt-get install gcc-multilib
displayName: Install gcc-multilib
- script: cargo install cross
displayName: Install cross
# Always patch
- template: azure-patch-crates.yml
- script: cross check --all --exclude tokio-tls --target $(target)
displayName: Check source
# - script: cross check --tests --all --exclude tokio-tls --target $(target)
# displayName: Check tests
+39
View File
@@ -0,0 +1,39 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
# rust_version: stable
rust_version: ${{ parameters.rust }}
- script: |
cargo doc --all --no-deps --all-features
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+33
View File
@@ -0,0 +1,33 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal --default-toolchain none
export PATH=$PATH:$HOME/.cargo/bin
rustup toolchain install $RUSTUP_TOOLCHAIN
rustup default $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --profile minimal --default-toolchain none
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
rustup toolchain install %RUSTUP_TOOLCHAIN%
rustup default %RUSTUP_TOOLCHAIN%
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustup toolchain list
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%B' | grep -qF '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+18
View File
@@ -0,0 +1,18 @@
jobs:
- job: ${{ parameters.name }}
displayName: Loom tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- ${{ each crate in parameters.crates }}:
- script: RUSTFLAGS="--cfg loom" cargo test --lib --release --features "full" -- --test-threads=1 --nocapture
env:
LOOM_MAX_PREEMPTIONS: 1
CI: 'True'
displayName: test ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+17
View File
@@ -0,0 +1,17 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: |
rustup component add rustfmt
cargo fmt --version
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+17
View File
@@ -0,0 +1,17 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: 'Ubuntu 16.04'
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- script: cargo install cargo-hack
displayName: Install cargo-hack
- script: cargo hack test --each-feature
displayName: cargo hack test --each-feature
workingDirectory: $(Build.SourcesDirectory)/tests-build
+19
View File
@@ -0,0 +1,19 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
# Check benches
- script: cargo check --benches --all
displayName: Check benchmarks
+42
View File
@@ -0,0 +1,42 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
# Run with all crate features
- script: cargo test --all-features
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate }} - cargo test --all-features
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
# Run with all crate features
- script: cargo test --all-features
env:
LOOM_MAX_PREEMPTIONS: 2
CI: 'True'
displayName: ${{ crate }} - cargo test --all-features
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+34
View File
@@ -0,0 +1,34 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+8
View File
@@ -0,0 +1,8 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-macros = { path = "tokio-macros" }
tokio-test = { path = "tokio-test" }
tokio-tls = { path = "tokio-tls" }
tokio-util = { path = "tokio-util" }
+16 -10
View File
@@ -3,26 +3,27 @@
# TSAN does not understand fences and `Arc::drop` is implemented using a fence.
# This causes many false positives.
race:Arc*drop
race:arc*Weak*drop
race:Weak*drop
# `std` mpsc is not used in any Tokio code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
race:std*lang_start
race:drop*std::thread*
# Probably more fences in std.
race:__call_tls_dtors
# The crossbeam deque uses fences.
race:crossbeam_deque
# The epoch-based GC uses fences.
race:crossbeam_epoch
# This is excluded as this race shows up due to using the stealing features of
# the deque. Unfortunately, the implementation uses a fence, which makes tsan
# unhappy.
#
# TODO: It would be nice to not have to filter this out.
race:try_steal_task
# Push and steal operations in crossbeam-deque may cause data races, but such
# data races are safe. If a data race happens, the value read by `steal` is
# forgotten and the steal operation is then retried.
race:crossbeam_deque*push
race:crossbeam_deque*steal
# This filters out expected data race in the treiber stack implementations.
# This filters out expected data race in the Treiber stack implementations.
# Treiber stacks are inherently racy. The pop operation will attempt to access
# the "next" pointer on the node it is attempting to pop. However, at this
# point it has not gained ownership of the node and another thread might beat
@@ -30,4 +31,9 @@ race:try_steal_task
# original pop operation will fail due to the ABA guard, but tsan still picks
# up the access on the next pointer.
race:Backup::next_sleeper
race:Backup::set_next_sleeper
race:WorkerEntry::set_next_sleeper
# This ignores a false positive caused by `thread::park()`/`thread::unpark()`.
# See: https://github.com/rust-lang/rust/pull/54806#issuecomment-436193353
race:pthread_cond_destroy
+61
View File
@@ -0,0 +1,61 @@
[package]
name = "examples"
version = "0.0.0"
publish = false
edition = "2018"
[dev-dependencies]
tokio = { version = "0.2.0", path = "../tokio", features = ["full"] }
tokio-util = { version = "0.2.0", path = "../tokio-util", features = ["full"] }
bytes = "0.5"
futures = "0.3.0"
http = "0.2"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
httparse = "1.0"
time = "0.1"
[[example]]
name = "chat"
path = "chat.rs"
[[example]]
name = "connect"
path = "connect.rs"
[[example]]
name = "echo-udp"
path = "echo-udp.rs"
[[example]]
name = "echo"
path = "echo.rs"
[[example]]
name = "hello_world"
path = "hello_world.rs"
[[example]]
name = "print_each_packet"
path = "print_each_packet.rs"
[[example]]
name = "proxy"
path = "proxy.rs"
[[example]]
name = "tinydb"
path = "tinydb.rs"
[[example]]
name = "udp-client"
path = "udp-client.rs"
[[example]]
name = "udp-codec"
path = "udp-codec.rs"
[[example]]
name = "tinyhttp"
path = "tinyhttp.rs"
+4 -58
View File
@@ -1,60 +1,6 @@
## Examples of how to use Tokio
This directory contains a number of examples showcasing various capabilities of
the `tokio` crate.
All examples can be executed with:
```
cargo run --example $name
```
A high level description of each example is:
* [`hello_world`](hello_world.rs) - a tiny server that writes "hello world" to
all connected clients and then terminates the connection, should help see how
to create and initialize `tokio`.
* [`echo`](echo.rs) - this is your standard TCP "echo server" which accepts
connections and then echos back any contents that are read from each connected
client.
* [`print_each_packet`](print_each_packet.rs) - this server will create a TCP
listener, accept connections in a loop, and put down in the stdout everything
that's read off of each TCP connection.
* [`echo-udp`](echo-udp.rs) - again your standard "echo server", except for UDP
instead of TCP. This will echo back any packets received to the original
sender.
* [`connect`](connect.rs) - this is a `nc`-like clone which can be used to
interact with most other examples. The program creates a TCP connection or UDP
socket to sends all information read on stdin to the remote peer, displaying
any data received on stdout. Often quite useful when interacting with the
various other servers here!
* [`chat`](chat.rs) - this spins up a local TCP server which will broadcast from
any connected client to all other connected clients. You can connect to this
in multiple terminals and use it to chat between the terminals.
* [`chat-combinator`](chat-combinator.rs) - Similar to `chat`, but this uses a
much more functional programming approach using combinators.
* [`proxy`](proxy.rs) - an example proxy server that will forward all connected
TCP clients to the remote address specified when starting the program.
* [`tinyhttp`](tinyhttp.rs) - a tiny HTTP/1.1 server which doesn't support HTTP
request bodies showcasing running on multiple cores, working with futures and
spawning tasks, and finally framing a TCP connection to discrete
request/response objects.
* [`tinydb`](tinydb.rs) - an in-memory database which shows sharing state
between all connected clients, notably the key/value store of this database.
* [`udp-client`](udp-client.rs) - a simple `send_dgram`/`recv_dgram` example.
* [`manual-runtime`](manual-runtime.rs) - manually composing a runtime.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
The `master` branch is currently being updated to use `async` / `await`.
The examples are not fully ported. Examples for stable Tokio can be
found
[here](https://github.com/tokio-rs/tokio/tree/v0.1.x/tokio/examples).
-150
View File
@@ -1,150 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate tokio;
extern crate futures;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::sync::{Arc, Mutex};
fn main() {
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse().unwrap();
let socket = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
// This is running on the Tokio runtime, so it will be multi-threaded. The
// `Arc<Mutex<...>>` allows state to be shared across the threads.
let connections = Arc::new(Mutex::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr().unwrap();
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
connections.lock().unwrap().insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.lock().unwrap();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn a task to process the connection
tokio::spawn(connection.then(move |_| {
connections.lock().unwrap().remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}));
Ok(())
});
// execute server
tokio::run(srv);
}
+158 -374
View File
@@ -24,29 +24,64 @@
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
#[macro_use]
extern crate futures;
extern crate bytes;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use tokio::stream::{Stream, StreamExt};
use tokio::sync::{mpsc, Mutex};
use tokio_util::codec::{Framed, LinesCodec, LinesCodecError};
use futures::SinkExt;
use std::collections::HashMap;
use std::env;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use std::pin::Pin;
use std::sync::Arc;
use std::task::{Context, Poll};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:6142".to_string());
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let mut listener = TcpListener::bind(&addr).await?;
println!("server running on {}", addr);
loop {
// Asynchronously wait for an inbound TcpStream.
let (stream, addr) = listener.accept().await?;
// Clone a handle to the `Shared` state for the new connection.
let state = Arc::clone(&state);
// Spawn our handler to be run asynchronously.
tokio::spawn(async move {
if let Err(e) = process(state, stream, addr).await {
println!("an error occured; error = {:?}", e);
}
});
}
}
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<Bytes>;
type Tx = mpsc::UnboundedSender<String>;
/// Shorthand for the receive half of the message channel.
type Rx = mpsc::UnboundedReceiver<Bytes>;
type Rx = mpsc::UnboundedReceiver<String>;
/// Data that is shared between all peers in the chat server.
///
@@ -60,64 +95,18 @@ struct Shared {
/// The state for each connected client.
struct Peer {
/// Name of the peer.
///
/// When a client connects, the first line sent is treated as the client's
/// name (like alice or bob). The name is used to preface all messages that
/// arrive from the client so that we can simulate a real chat server:
///
/// ```text
/// alice: Hello everyone.
/// bob: Welcome to telnet chat!
/// ```
name: BytesMut,
/// The TCP socket wrapped with the `Lines` codec, defined below.
///
/// This handles sending and receiving data on the socket. When using
/// `Lines`, we can work at the line level instead of having to manage the
/// raw byte operations.
lines: Lines,
/// Handle to the shared chat state.
///
/// This is used to broadcast messages read off the socket to all connected
/// peers.
state: Arc<Mutex<Shared>>,
lines: Framed<TcpStream, LinesCodec>,
/// Receive half of the message channel.
///
/// This is used to receive messages from peers. When a message is received
/// off of this `Rx`, it will be written to the socket.
rx: Rx,
/// Client socket address.
///
/// The socket address is used as the key in the `peers` HashMap. The
/// address is saved so that the `Peer` drop implementation can clean up its
/// entry.
addr: SocketAddr,
}
/// Line based codec
///
/// This decorates a socket and presents a line based read / write interface.
///
/// As a user of `Lines`, we can focus on working at the line level. So, we send
/// and receive values that represent entire lines. The `Lines` codec will
/// handle the encoding and decoding as well as reading from and writing to the
/// socket.
#[derive(Debug)]
struct Lines {
/// The TCP socket.
socket: TcpStream,
/// Buffer used when reading from the socket. Data is not returned from this
/// buffer until an entire line has been read.
rd: BytesMut,
/// Buffer used to stage data before writing it to the socket.
wr: BytesMut,
}
impl Shared {
@@ -127,348 +116,143 @@ impl Shared {
peers: HashMap::new(),
}
}
/// Send a `LineCodec` encoded message to every peer, except
/// for the sender.
async fn broadcast(&mut self, sender: SocketAddr, message: &str) {
for peer in self.peers.iter_mut() {
if *peer.0 != sender {
let _ = peer.1.send(message.into());
}
}
}
}
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
async fn new(
state: Arc<Mutex<Shared>>,
lines: Framed<TcpStream, LinesCodec>,
) -> io::Result<Peer> {
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
let addr = lines.get_ref().peer_addr()?;
// Create a channel for this peer
let (tx, rx) = mpsc::unbounded();
let (tx, rx) = mpsc::unbounded_channel();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
state.lock().await.peers.insert(addr, tx);
Peer {
name,
lines,
state,
rx,
addr,
}
Ok(Peer { lines, rx })
}
}
/// This is where a connected client is managed.
///
/// A `Peer` is also a future representing completely processing the client.
///
/// When a `Peer` is created, the first line (representing the client's name)
/// has already been read. When the socket closes, the `Peer` future completes.
///
/// While processing, the peer future implementation will:
///
/// 1) Receive messages on its message channel and write them to the socket.
/// 2) Receive messages from the socket and broadcast them to all peers.
///
impl Future for Peer {
type Item = ();
type Error = io::Error;
#[derive(Debug)]
enum Message {
/// A message that should be broadcasted to others.
Broadcast(String),
fn poll(&mut self) -> Poll<(), io::Error> {
// Tokio (and futures) use cooperative scheduling without any
// preemption. If a task never yields execution back to the executor,
// then other tasks may be starved.
//
// To deal with this, robust applications should not have any unbounded
// loops. In this example, we will read at most `LINES_PER_TICK` lines
// from the client on each tick.
//
// If the limit is hit, the current task is notified, informing the
// executor to schedule the task again asap.
const LINES_PER_TICK: usize = 10;
// Receive all messages from peers.
for i in 0..LINES_PER_TICK {
// Polling an `UnboundedReceiver` cannot fail, so `unwrap` here is
// safe.
match self.rx.poll().unwrap() {
Async::Ready(Some(v)) => {
// Buffer the line. Once all lines are buffered, they will
// be flushed to the socket (right below).
self.lines.buffer(&v);
// If this is the last iteration, the loop will break even
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
task::current().notify();
}
}
_ => break,
}
}
// Flush the write buffer to the socket
let _ = self.lines.poll_flush()?;
// Read new lines from the socket
while let Async::Ready(line) = self.lines.poll()? {
println!("Received line ({:?}) : {:?}", self.name, line);
if let Some(message) = line {
// Append the peer's name to the front of the line:
let mut line = self.name.clone();
line.extend_from_slice(b": ");
line.extend_from_slice(&message);
line.extend_from_slice(b"\r\n");
// We're using `Bytes`, which allows zero-copy clones (by
// storing the data in an Arc internally).
//
// However, before cloning, we must freeze the data. This
// converts it from mutable -> immutable, allowing zero copy
// cloning.
let line = line.freeze();
// Now, send the line to all other peers
for (addr, tx) in &self.state.lock().unwrap().peers {
// Don't send the message to ourselves
if *addr != self.addr {
// The send only fails if the rx half has been dropped,
// however this is impossible as the `tx` half will be
// removed from the map before the `rx` is dropped.
tx.unbounded_send(line.clone()).unwrap();
}
}
} else {
// EOF was reached. The remote client has disconnected. There is
// nothing more to do.
return Ok(Async::Ready(()));
}
}
// As always, it is important to not just return `NotReady` without
// ensuring an inner future also returned `NotReady`.
//
// We know we got a `NotReady` from either `self.rx` or `self.lines`, so
// the contract is respected.
Ok(Async::NotReady)
}
/// A message that should be received by a client
Received(String),
}
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
}
}
// Peer implements `Stream` in a way that polls both the `Rx`, and `Framed` types.
// A message is produced whenever an event is ready until the `Framed` stream returns `None`.
impl Stream for Peer {
type Item = Result<Message, LinesCodecError>;
impl Lines {
/// Create a new `Lines` codec backed by the socket
fn new(socket: TcpStream) -> Self {
Lines {
socket,
rd: BytesMut::new(),
wr: BytesMut::new(),
}
}
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// First poll the `UnboundedReceiver`.
/// Buffer a line.
///
/// This writes the line to an internal buffer. Calls to `poll_flush` will
/// attempt to flush this buffer to the socket.
fn buffer(&mut self, line: &[u8]) {
// Ensure the buffer has capacity. Ideally this would not be unbounded,
// but to keep the example simple, we will not limit this.
self.wr.reserve(line.len());
// Push the line onto the end of the write buffer.
//
// The `put` function is from the `BufMut` trait.
self.wr.put(line);
}
/// Flush the write buffer to the socket
fn poll_flush(&mut self) -> Poll<(), io::Error> {
// As long as there is buffered data to write, try to write it.
while !self.wr.is_empty() {
// Try to read some bytes from the socket
let n = try_ready!(self.socket.poll_write(&self.wr));
// As long as the wr is not empty, a successful write should
// never write 0 bytes.
assert!(n > 0);
// This discards the first `n` bytes of the buffer.
let _ = self.wr.split_to(n);
if let Poll::Ready(Some(v)) = Pin::new(&mut self.rx).poll_next(cx) {
return Poll::Ready(Some(Ok(Message::Received(v))));
}
Ok(Async::Ready(()))
}
// Secondly poll the `Framed` stream.
let result: Option<_> = futures::ready!(Pin::new(&mut self.lines).poll_next(cx));
/// Read data from the socket.
///
/// This only returns `Ready` when the socket has closed.
fn fill_read_buf(&mut self) -> Poll<(), io::Error> {
loop {
// Ensure the read buffer has capacity.
//
// This might result in an internal allocation.
self.rd.reserve(1024);
Poll::Ready(match result {
// We've received a message we should broadcast to others.
Some(Ok(message)) => Some(Ok(Message::Broadcast(message))),
// Read data into the buffer.
let n = try_ready!(self.socket.read_buf(&mut self.rd));
// An error occured.
Some(Err(e)) => Some(Err(e)),
if n == 0 {
return Ok(Async::Ready(()));
}
}
}
}
impl Stream for Lines {
type Item = BytesMut;
type Error = io::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
// First, read any new data that might have been received off the socket
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
if let Some(pos) = pos {
// Remove the line from the read buffer and set it to `line`.
let mut line = self.rd.split_to(pos + 2);
// Drop the trailing \r\n
line.split_off(pos);
// Return the line
return Ok(Async::Ready(Some(line)));
}
if sock_closed {
Ok(Async::Ready(None))
} else {
Ok(Async::NotReady)
}
}
}
/// Spawn a task to manage the socket.
///
/// This will read the first line from the socket to identify the client, then
/// add the client to the set of connected peers in the chat service.
fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// Wrap the socket with the `Lines` codec that we wrote above.
//
// By doing this, we can operate at the line level instead of doing raw byte
// manipulation.
let lines = Lines::new(socket);
// The first line is treated as the client's name. The client is not added
// to the set of connected peers until this line is received.
//
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
// Process the first received line as the client's name.
.and_then(|(name, lines)| {
// If `name` is `None`, then the client disconnected without
// actually sending a line of data.
//
// Since the connection is closed, there is no further work that we
// need to do. So, we just terminate processing by returning
// `future::ok()`.
//
// The problem is that only a single future type can be returned
// from a combinator closure, but we want to return both
// `future::ok()` and `Peer` (below).
//
// This is a common problem, so the `futures` crate solves this by
// providing the `Either` helper enum that allows creating a single
// return type that covers two concrete future types.
let name = match name {
Some(name) => name,
None => {
// The remote client closed the connection without sending
// any data.
return Either::A(future::ok(()));
}
};
println!("`{:?}` is joining the chat", name);
// Create the peer.
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
// The stream has been exhausted.
None => None,
})
// Task futures have an error of type `()`, this ensures we handle the
// error. We do this by printing the error to STDOUT.
.map_err(|e| {
println!("connection error = {:?}", e);
});
// Spawn the task. Internally, this submits the task to a thread pool.
tokio::spawn(connection);
}
}
pub fn main() {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
/// Process an individual chat client
async fn process(
state: Arc<Mutex<Shared>>,
stream: TcpStream,
addr: SocketAddr,
) -> Result<(), Box<dyn Error>> {
let mut lines = Framed::new(stream, LinesCodec::new());
let addr = "127.0.0.1:6142".parse().unwrap();
// Send a prompt to the client to enter their username.
lines
.send(String::from("Please enter your username:"))
.await?;
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
// Read the first line from the `LineCodec` stream to get the username.
let username = match lines.next().await {
Some(Ok(line)) => line,
// We didn't get a line so we return early here.
_ => {
println!("Failed to get username from {}. Client disconnected.", addr);
return Ok(());
}
};
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
// Register our peer with state which internally sets up some channels.
let mut peer = Peer::new(state.clone(), lines).await?;
println!("server running on localhost:6142");
// A client has connected, let's let everyone know.
{
let mut state = state.lock().await;
let msg = format!("{} has joined the chat", username);
println!("{}", msg);
state.broadcast(addr, &msg).await;
}
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
// Process incoming messages until our stream is exhausted by a disconnect.
while let Some(result) = peer.next().await {
match result {
// A message was received from the current user, we should
// broadcast this message to the other users.
Ok(Message::Broadcast(msg)) => {
let mut state = state.lock().await;
let msg = format!("{}: {}", username, msg);
state.broadcast(addr, &msg).await;
}
// A message was received from a peer. Send it to the
// current user.
Ok(Message::Received(msg)) => {
peer.lines.send(msg).await?;
}
Err(e) => {
println!(
"an error occured while processing messages for {}; error = {:?}",
username, e
);
}
}
}
// If this section is reached it means that the client was disconnected!
// Let's let everyone still connected know about it.
{
let mut state = state.lock().await;
state.peers.remove(&addr);
let msg = format!("{} has left the chat", username);
println!("{}", msg);
state.broadcast(addr, &msg).await;
}
Ok(())
}
+114 -174
View File
@@ -14,23 +14,17 @@
//! this repository! Many of them recommend running this as a simple "hook up
//! stdin/stdout to a server" to get up and running.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use tokio::io;
use tokio_util::codec::{FramedRead, FramedWrite};
use std::env;
use std::io::{self, Read, Write};
use std::error::Error;
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
fn main() {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
@@ -42,48 +36,120 @@ fn main() {
};
// Parse what address we're going to connect to
let addr = args.first().unwrap_or_else(|| {
panic!("this program requires at least one argument")
});
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = args
.first()
.ok_or("this program requires at least one argument")?;
let addr = addr.parse::<SocketAddr>()?;
// Right now Tokio doesn't support a handle to stdin running on the event
// loop, so we farm out that work to a separate thread. This thread will
// read data (with blocking I/O) from stdin and then send it to the event
// loop over a standard futures channel.
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(|| read_stdin(stdin_tx));
let stdin_rx = stdin_rx.map_err(|_| panic!()); // errors not possible on rx
let stdin = FramedRead::new(io::stdin(), codec::Bytes);
let stdout = FramedWrite::new(io::stdout(), codec::Bytes);
// Now that we've got our stdin read we either set up our TCP connection or
// our UDP connection to get a stream of bytes we're going to emit to
// stdout.
let stdout = if tcp {
tcp::connect(&addr, Box::new(stdin_rx))
if tcp {
tcp::connect(&addr, stdin, stdout).await?;
} else {
udp::connect(&addr, Box::new(stdin_rx))
};
udp::connect(&addr, stdin, stdout).await?;
}
// And now with our stream of bytes to write to stdout, we execute that in
// the event loop! Note that this is doing blocking I/O to emit data to
// stdout, and in general it's a no-no to do that sort of work on the event
// loop. In this case, though, we know it's ok as the event loop isn't
// otherwise running anything useful.
let mut out = io::stdout();
Ok(())
}
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
mod tcp {
use super::codec;
use futures::StreamExt;
use futures::{future, Sink, SinkExt};
use std::{error::Error, io, net::SocketAddr};
use tokio::net::TcpStream;
use tokio::stream::Stream;
use tokio_util::codec::{FramedRead, FramedWrite};
pub async fn connect(
addr: &SocketAddr,
mut stdin: impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin,
mut stdout: impl Sink<Vec<u8>, Error = io::Error> + Unpin,
) -> Result<(), Box<dyn Error>> {
let mut stream = TcpStream::connect(addr).await?;
let (r, w) = stream.split();
let mut sink = FramedWrite::new(w, codec::Bytes);
let mut stream = FramedRead::new(r, codec::Bytes)
.filter_map(|i| match i {
Ok(i) => future::ready(Some(i)),
Err(e) => {
println!("failed to read from socket; error={}", e);
future::ready(None)
}
})
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
.map(Ok);
match future::join(sink.send_all(&mut stdin), stdout.send_all(&mut stream)).await {
(Err(e), _) | (_, Err(e)) => Err(e.into()),
_ => Ok(()),
}
}
}
mod udp {
use tokio::net::udp::{RecvHalf, SendHalf};
use tokio::net::UdpSocket;
use tokio::stream::{Stream, StreamExt};
use futures::{future, Sink, SinkExt};
use std::error::Error;
use std::io;
use std::net::SocketAddr;
pub async fn connect(
addr: &SocketAddr,
stdin: impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin,
stdout: impl Sink<Vec<u8>, Error = io::Error> + Unpin,
) -> Result<(), Box<dyn Error>> {
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let bind_addr = if addr.ip().is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
};
let socket = UdpSocket::bind(&bind_addr).await?;
socket.connect(addr).await?;
let (mut r, mut w) = socket.split();
future::try_join(send(stdin, &mut w), recv(stdout, &mut r)).await?;
Ok(())
}
async fn send(
mut stdin: impl Stream<Item = Result<Vec<u8>, io::Error>> + Unpin,
writer: &mut SendHalf,
) -> Result<(), io::Error> {
while let Some(item) = stdin.next().await {
let buf = item?;
writer.send(&buf[..]).await?;
}
Ok(())
}
async fn recv(
mut stdout: impl Sink<Vec<u8>, Error = io::Error> + Unpin,
reader: &mut RecvHalf,
) -> Result<(), io::Error> {
loop {
let mut buf = vec![0; 1024];
let n = reader.recv(&mut buf[..]).await?;
if n > 0 {
stdout.send(buf).await?;
}
}
}
}
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio_codec::{Encoder, Decoder};
use std::io;
use tokio_util::codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
///
@@ -95,13 +161,13 @@ mod codec {
pub struct Bytes;
impl Decoder for Bytes {
type Item = BytesMut;
type Item = Vec<u8>;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() > 0 {
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<Vec<u8>>> {
if !buf.is_empty() {
let len = buf.len();
Ok(Some(buf.split_to(len)))
Ok(Some(buf.split_to(len).into_iter().collect()))
} else {
Ok(None)
}
@@ -118,129 +184,3 @@ mod codec {
}
}
}
mod tcp {
use tokio;
use tokio_codec::Decoder;
use tokio::net::TcpStream;
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
// to start forwarding data.
//
// First we use the `Io::framed` method with a simple implementation of
// a `Codec` (listed below) that just ships bytes around. We then split
// that in two to work with the stream and sink separately.
//
// Half of the work we're going to do is to take all data we receive on
// `stdin` and send that along the TCP stream (`sink`). The second half
// is to take all the data we receive (`stream`) and then write that to
// stdout. We'll be passing this handle back out from this method.
//
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
Box::new(tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream())
}
}
mod udp {
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
"0.0.0.0:0".parse().unwrap()
} else {
"[::]:0".parse().unwrap()
};
let udp = UdpSocket::bind(&addr_to_bind)
.expect("failed to bind socket");
// Like above with TCP we use an instance of `Bytes` codec to transform
// this UDP socket into a framed sink/stream which operates over
// discrete values. In this case we're working with *pairs* of socket
// addresses and byte buffers.
let (sink, stream) = UdpFramed::new(udp, Bytes).split();
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
let forward_stdin = stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
let receive = stream.filter_map(move |(chunk, src)| {
if src == addr {
Some(chunk.into())
} else {
None
}
});
Box::new(future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
}).flatten_stream())
}
}
// Our helper method which will read data from stdin and send it along the
// sender provided.
fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
let mut stdin = io::stdin();
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
tx = match tx.send(buf).wait() {
Ok(tx) => tx,
Err(_) => break,
};
}
}
+27 -29
View File
@@ -1,6 +1,6 @@
//! An UDP echo server that just sends back everything that it receives.
//!
//! If you're on unix you can test this out by in one terminal executing:
//! If you're on Unix you can test this out by in one terminal executing:
//!
//! cargo run --example echo-udp
//!
@@ -10,16 +10,12 @@
//!
//! Each line you type in to the `nc` terminal should be echo'd back to you!
#![deny(warnings)]
#![warn(rust_2018_idioms)]
#[macro_use]
extern crate futures;
extern crate tokio;
use std::{env, io};
use std::error::Error;
use std::net::SocketAddr;
use tokio::prelude::*;
use std::{env, io};
use tokio;
use tokio::net::UdpSocket;
struct Server {
@@ -28,46 +24,48 @@ struct Server {
to_send: Option<(usize, SocketAddr)>,
}
impl Future for Server {
type Item = ();
type Error = io::Error;
impl Server {
async fn run(self) -> Result<(), io::Error> {
let Server {
mut socket,
mut buf,
mut to_send,
} = self;
fn poll(&mut self) -> Poll<(), io::Error> {
loop {
// First we check to see if there's a message we need to echo back.
// If so then we try to send it back to the original source, waiting
// until it's writable and we're able to do so.
if let Some((size, peer)) = self.to_send {
let amt = try_ready!(self.socket.poll_send_to(&self.buf[..size], &peer));
if let Some((size, peer)) = to_send {
let amt = socket.send_to(&buf[..size], &peer).await?;
println!("Echoed {}/{} bytes to {}", amt, size, peer);
self.to_send = None;
}
// If we're here then `to_send` is `None`, so we take a look for the
// next message we're going to echo back.
self.to_send = Some(try_ready!(self.socket.poll_recv_from(&mut self.buf)));
to_send = Some(socket.recv_from(&mut buf).await?);
}
}
}
fn main() {
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
let socket = UdpSocket::bind(&addr).unwrap();
println!("Listening on: {}", socket.local_addr().unwrap());
let socket = UdpSocket::bind(&addr).await?;
println!("Listening on: {}", socket.local_addr()?);
let server = Server {
socket: socket,
socket,
buf: vec![0; 1024],
to_send: None,
};
// This starts the server task.
//
// `map_err` handles the error by logging it and maps the future to a type
// that can be spawned.
//
// `tokio::run` spawns the task on the Tokio runtime and starts running.
tokio::run(server.map_err(|e| println!("server error = {:?}", e)));
server.run().await?;
Ok(())
}
+40 -75
View File
@@ -19,96 +19,61 @@
//! you! If you open up multiple terminals running the `connect` example you
//! should be able to see them all make progress simultaneously.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use tokio::io;
use tokio;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
use std::error::Error;
fn main() {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
// above and must be associated with an event loop.
let mut listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We just want to copy all data read from the socket back onto the
// socket itself (e.g. "echo"). We can use the standard `io::copy`
// combinator in the `tokio-core` crate to do precisely this!
//
// The `copy` function takes two arguments, where to read from and where
// to write to. We only have one argument, though, with `socket`.
// Luckily there's a method, `Io::split`, which will split an Read/Write
// stream into its two halves. This operation allows us to work with
// each stream independently, such as pass them as two arguments to the
// `copy` function.
//
// The `copy` function then returns a future, and this future will be
// resolved when the copying operation is complete, resolving to the
// amount of data that was copied.
let (reader, writer) = socket.split();
let amt = io::copy(reader, writer);
loop {
// Asynchronously wait for an inbound socket.
let (mut socket, _) = listener.accept().await?;
// After our copy operation is complete we just print out some helpful
// information.
let msg = amt.then(move |result| {
match result {
Ok((amt, _, _)) => println!("wrote {} bytes", amt),
Err(e) => println!("error: {}", e),
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(async move {
let mut buf = [0; 1024];
// In a loop, read data from the socket and write the data back.
loop {
let n = socket
.read(&mut buf)
.await
.expect("failed to read data from socket");
if n == 0 {
return;
}
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(msg)
socket
.write_all(&buf[0..n])
.await
.expect("failed to write data to socket");
}
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
}
}
+16 -53
View File
@@ -1,70 +1,33 @@
//! Hello world server.
//!
//! A simple server that accepts connections, writes "hello world\n", and closes
//! A simple client that opens a TCP stream, writes "hello world\n", and closes
//! the connection.
//!
//! You can test this out by running:
//!
//! cargo run --example hello_world
//! ncat -l 6142
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
//! cargo run --example hello_world
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpStream;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::error::Error;
pub fn main() {
let addr = "127.0.0.1:6142".parse().unwrap();
// Bind a TCP listener to the socket address.
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn Error>> {
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
// Note that this is the Tokio TcpStream, which is fully async.
let mut stream = TcpStream::connect("127.0.0.1:6142").await?;
println!("created stream");
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(|socket| {
println!("accepted socket; addr={:?}", socket.peer_addr().unwrap());
let result = stream.write(b"hello world\n").await;
println!("wrote to stream; success={:?}", result.is_ok());
let connection = io::write_all(socket, "hello world\n")
.then(|res| {
println!("wrote message; success={:?}", res.is_ok());
Ok(())
});
// Spawn a new task that processes the socket:
tokio::spawn(connection);
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
// Start the Tokio runtime.
//
// The Tokio is a pre-configured "out of the box" runtime for building
// asynchronous applications. It includes both a reactor and a task
// scheduler. This means applications are multithreaded by default.
//
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
Ok(())
}
-85
View File
@@ -1,85 +0,0 @@
//! An example how to manually assemble a runtime and run some tasks on it.
//!
//! This is closer to the single-threaded runtime than the default tokio one, as it is simpler to
//! grasp. There are conceptually similar, but the multi-threaded one would be more code. If you
//! just want to *use* a single-threaded runtime, use the one provided by tokio directly
//! (`tokio::runtime::current_thread::Runtime::new()`. This is a demonstration only.
//!
//! Note that the error handling is a bit left out. Also, the `run` could be modified to return the
//! result of the provided future.
extern crate futures;
extern crate tokio;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_timer;
use std::io::Error as IoError;
use std::time::{Duration, Instant};
use futures::{future, Future};
use tokio::executor::current_thread::{self, CurrentThread};
use tokio_reactor::Reactor;
use tokio_timer::timer::{self, Timer};
/// Creates a „runtime“.
///
/// This is similar to running `tokio::runtime::current_thread::Runtime::new()`.
fn run<F: Future<Item = (), Error = ()>>(f: F) -> Result<(), IoError> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new(reactor);
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor generate some new stimuli for the
// futures to continue in their life.
let mut executor = CurrentThread::new_with_park(timer);
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the current single-threaded
// executor when used. This is a trick, because we need two mutable references to the
// executor (one to run the provided future, another to install as the default one). We
// use the fake one here as the default one.
let mut default_executor = current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
// Run the provided future
executor.block_on(f).unwrap();
// Run all the other futures that are still left in the executor
executor.run().unwrap();
});
});
});
Ok(())
}
fn main() {
run(future::lazy(|| {
// Here comes the application logic. It can spawn further tasks by current_thread::spawn().
// It also can use the default reactor and create timeouts.
// Connect somewhere. And then do nothing with it. Yes, useless.
//
// This will use the default reactor which runs in the current thread.
let connect = tokio::net::TcpStream::connect(&"127.0.0.1:53".parse().unwrap())
.map(|_| println!("Connected"))
.map_err(|e| println!("Failed to connect: {}", e));
// We can spawn it without requiring Send. This would panic if we run it outside of the
// `run` (or outside of anything else)
current_thread::spawn(connect);
// We can also create timeouts.
let deadline = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(5))
.map(|()| println!("5 seconds are over"))
.map_err(|e| println!("Failed to wait: {}", e));
// We can spawn on the default executor, which is also the local one.
tokio::executor::spawn(deadline);
Ok(())
})).unwrap();
}
+32 -75
View File
@@ -52,98 +52,55 @@
//! ```
//!
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_codec::{Decoder, BytesCodec};
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::stream::StreamExt;
use tokio_util::codec::{BytesCodec, Decoder};
use std::env;
use std::net::SocketAddr;
fn main() {
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
let mut listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We're parsing each socket with the `BytesCodec` included in `tokio_io`,
// and then we `split` each codec into the reader/writer halves.
//
// See https://docs.rs/tokio-codec/0.1/src/tokio_codec/bytes_codec.rs.html
let framed = BytesCodec::new().framed(socket);
let (_writer, reader) = framed.split();
loop {
// Asynchronously wait for an inbound socket.
let (socket, _) = listener.accept().await?;
let processor = reader
.for_each(|bytes| {
println!("bytes: {:?}", bytes);
Ok(())
})
// After our copy operation is complete we just print out some helpful
// information.
.and_then(|()| {
println!("Socket received FIN packet and closed connection");
Ok(())
})
.or_else(|err| {
println!("Socket closed with error: {:?}", err);
// We have to return the error to catch it in the next ``.then` call
Err(err)
})
.then(|result| {
println!("Socket closed with result: {:?}", result);
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(async move {
// We're parsing each socket with the `BytesCodec` included in `tokio::codec`.
let mut framed = BytesCodec::new().framed(socket);
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(processor)
// We loop while there are messages coming from the Stream `framed`.
// The stream will return None once the client disconnects.
while let Some(message) = framed.next().await {
match message {
Ok(bytes) => println!("bytes: {:?}", bytes),
Err(err) => println!("Socket closed with error: {:?}", err),
}
}
println!("Socket received FIN packet and closed connection");
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
}
}
+33 -89
View File
@@ -20,109 +20,53 @@
//! This final terminal will connect to our proxy, which will in turn connect to
//! the echo server, and you'll be able to see data flowing between them.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use tokio::io::{copy, shutdown};
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
fn main() {
let listen_addr = env::args().nth(1).unwrap_or("127.0.0.1:8081".to_string());
let listen_addr = listen_addr.parse::<SocketAddr>().unwrap();
use futures::future::try_join;
use futures::FutureExt;
use std::env;
use std::error::Error;
let server_addr = env::args().nth(2).unwrap_or("127.0.0.1:8080".to_string());
let server_addr = server_addr.parse::<SocketAddr>().unwrap();
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let listen_addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8081".to_string());
let server_addr = env::args()
.nth(2)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
// Create a TCP listener which will listen for incoming connections.
let socket = TcpListener::bind(&listen_addr).unwrap();
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
let amounts = server.and_then(move |server| {
// Create separate read/write handles for the TCP clients that we're
// proxying data between. Note that typically you'd use
// `AsyncRead::split` for this operation, but we want our writer
// handles to have a custom implementation of `shutdown` which
// actually calls `TcpStream::shutdown` to ensure that EOF is
// transmitted properly across the proxied connection.
//
// As a result, we wrap up our client/server manually in arcs and
// use the impls below on our custom `MyTcpStream` type.
let client_reader = MyTcpStream(Arc::new(Mutex::new(client)));
let client_writer = client_reader.clone();
let server_reader = MyTcpStream(Arc::new(Mutex::new(server)));
let server_writer = server_reader.clone();
let mut listener = TcpListener::bind(listen_addr).await?;
// Copy the data (in parallel) between the client and the server.
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
Ok(())
while let Ok((inbound, _)) = listener.accept().await {
let transfer = transfer(inbound, server_addr.clone()).map(|r| {
if let Err(e) = r {
println!("Failed to transfer; error={}", e);
}
});
tokio::run(done);
}
// This is a custom type used to have a custom implementation of the
// `AsyncWrite::shutdown` method which actually calls `TcpStream::shutdown` to
// notify the remote end that we're done writing.
#[derive(Clone)]
struct MyTcpStream(Arc<Mutex<TcpStream>>);
impl Read for MyTcpStream {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
self.0.lock().unwrap().read(buf)
}
}
impl Write for MyTcpStream {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().unwrap().write(buf)
tokio::spawn(transfer);
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
Ok(())
}
impl AsyncRead for MyTcpStream {}
async fn transfer(mut inbound: TcpStream, proxy_addr: String) -> Result<(), Box<dyn Error>> {
let mut outbound = TcpStream::connect(proxy_addr).await?;
impl AsyncWrite for MyTcpStream {
fn shutdown(&mut self) -> Poll<(), io::Error> {
try!(self.0.lock().unwrap().shutdown(Shutdown::Write));
Ok(().into())
}
let (mut ri, mut wi) = inbound.split();
let (mut ro, mut wo) = outbound.split();
let client_to_server = io::copy(&mut ri, &mut wo);
let server_to_client = io::copy(&mut ro, &mut wi);
try_join(client_to_server, server_to_client).await?;
Ok(())
}
+112 -94
View File
@@ -39,24 +39,22 @@
//! * `SET $key $value` - this will set the value of `$key` to `$value`,
//! returning the previous value, if any.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
use tokio::io::{lines, write_all};
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::stream::StreamExt;
use tokio_util::codec::{Framed, LinesCodec};
use futures::SinkExt;
use std::collections::HashMap;
use std::env;
use std::error::Error;
use std::sync::{Arc, Mutex};
/// The in-memory database shared amongst all clients.
///
/// This database will be shared via `Arc`, so to mutate the internal map we're
/// also going to use a `RefCell` for interior mutability.
/// going to use a `Mutex` for interior mutability.
struct Database {
map: Mutex<HashMap<String, String>>,
}
@@ -69,17 +67,29 @@ enum Request {
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
Value {
key: String,
value: String,
},
Set {
key: String,
value: String,
previous: Option<String>,
},
Error {
msg: String,
},
}
fn main() {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let listener = TcpListener::bind(&addr).expect("failed to bind");
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
let mut listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
@@ -93,98 +103,108 @@ fn main() {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
// *split* this TCP stream into two separately owned halves. This'll
// allow us to work with the read and write halves independently.
let (reader, writer) = socket.split();
loop {
match listener.accept().await {
Ok((socket, _)) => {
// After getting a new connection first we see a clone of the database
// being created, which is creating a new reference for this connected
// client to use.
let db = db.clone();
// Since our protocol is line-based we use `tokio_io`'s `lines` utility
// to convert our stream of bytes, `reader`, into a `Stream` of lines.
let lines = lines(BufReader::new(reader));
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients. The `move` keyword is used
// here to move ownership of our db handle into the async closure.
tokio::spawn(async move {
// Since our protocol is line-based we use `tokio_codecs`'s `LineCodec`
// to convert our stream of bytes, `socket`, into a `Stream` of lines
// as well as convert our line based responses into a stream of bytes.
let mut lines = Framed::new(socket, LinesCodec::new());
// Here's where the meat of the processing in this server happens. First
// we see a clone of the database being created, which is creating a
// new reference for this connected client to use. Also note the `move`
// keyword on the closure here which moves ownership of the reference
// into the closure, which we'll need for spawning the client below.
//
// The `map` function here means that we'll run some code for all
// requests (lines) we receive from the client. The actual handling here
// is pretty simple, first we parse the request and if it's valid we
// generate a response based on the values in the database.
let db = db.clone();
let responses = lines.map(move |line| {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
// Here for every line we get back from the `Framed` decoder,
// we parse the request, and if it's valid we generate a response
// based on the values in the database.
while let Some(result) = lines.next().await {
match result {
Ok(line) => {
let response = handle_request(&line, &db);
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
let response = response.serialize();
if let Err(e) = lines.send(response).await {
println!("error on sending response; error = {:?}", e);
}
}
Err(e) => {
println!("error on decoding from socket; error = {:?}", e);
}
}
}
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
}
}
});
// At this point `responses` is a stream of `Response` types which we
// now want to write back out to the client. To do that we use
// `Stream::fold` to perform a loop here, serializing each response and
// then writing it out to the client.
let writes = responses.fold(writer, |writer, response| {
let mut response = response.serialize();
response.push('\n');
write_all(writer, response.into_bytes()).map(|(w, _)| w)
});
// The connection will be closed at this point as `lines.next()` has returned `None`.
});
}
Err(e) => println!("error accepting socket; error = {:?}", e),
}
}
}
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients, for now ignoring any errors
// that we see.
let msg = writes.then(move |_| Ok(()));
fn handle_request(line: &str, db: &Arc<Database>) -> Response {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
tokio::spawn(msg)
});
tokio::run(done);
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => match db.get(&key) {
Some(value) => Response::Value {
key,
value: value.clone(),
},
None => Response::Error {
msg: format!("no key {}", key),
},
},
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set {
key,
value,
previous,
}
}
}
}
impl Request {
fn parse(input: &str) -> Result<Request, String> {
let mut parts = input.splitn(3, " ");
let mut parts = input.splitn(3, ' ');
match parts.next() {
Some("GET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err(format!("GET must be followed by a key")),
};
let key = parts.next().ok_or("GET must be followed by a key")?;
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
return Err("GET's key must not be followed by anything".into());
}
Ok(Request::Get { key: key.to_string() })
Ok(Request::Get {
key: key.to_string(),
})
}
Some("SET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err(format!("SET must be followed by a key")),
None => return Err("SET must be followed by a key".into()),
};
let value = match parts.next() {
Some(value) => value,
None => return Err(format!("SET needs a value")),
None => return Err("SET needs a value".into()),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
Ok(Request::Set {
key: key.to_string(),
value: value.to_string(),
})
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
None => Err("empty input".into()),
}
}
}
@@ -192,15 +212,13 @@ impl Request {
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
Response::Value { ref key, ref value } => format!("{} = {}", key, value),
Response::Set {
ref key,
ref value,
ref previous,
} => format!("set {} = `{}`, previous: {:?}", key, value, previous),
Response::Error { ref msg } => format!("error: {}", msg),
}
}
}
+92 -97
View File
@@ -1,9 +1,9 @@
//! A "tiny" example of HTTP request/response handling using just tokio-core
//! A "tiny" example of HTTP request/response handling using transports.
//!
//! This example is intended for *learning purposes* to see how various pieces
//! hook up together and how HTTP can get up and running. Note that this example
//! is written with the restriction that it *can't* use any "big" library other
//! than tokio-core, if you'd like a "real world" HTTP library you likely want a
//! than Tokio, if you'd like a "real world" HTTP library you likely want a
//! crate like Hyper.
//!
//! Code here is based on the `echo-threads` example and implements two paths,
@@ -11,103 +11,85 @@
//! respectively. By default this will run I/O on all the cores your system has
//! available, and it doesn't support HTTP request bodies.
#![deny(warnings)]
extern crate bytes;
extern crate http;
extern crate httparse;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
extern crate time;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
use std::{env, fmt, io};
use std::net::SocketAddr;
use tokio::net::{TcpStream, TcpListener};
use tokio::prelude::*;
use tokio_codec::{Encoder, Decoder};
#![warn(rust_2018_idioms)]
use bytes::BytesMut;
use http::header::HeaderValue;
use http::{Request, Response, StatusCode};
use futures::SinkExt;
use http::{header::HeaderValue, Request, Response, StatusCode};
#[macro_use]
extern crate serde_derive;
use serde_json;
use std::{env, error::Error, fmt, io};
use tokio::net::{TcpListener, TcpStream};
use tokio::stream::StreamExt;
use tokio_util::codec::{Decoder, Encoder, Framed};
fn main() {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Parse the arguments, bind the TCP socket we'll be listening to, spin up
// our worker threads, and start shipping sockets to those worker threads.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let listener = TcpListener::bind(&addr).expect("failed to bind");
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
let mut server = TcpListener::bind(&addr).await?;
let mut incoming = server.incoming();
println!("Listening on: {}", addr);
tokio::run({
listener.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(|socket| {
process(socket);
Ok(())
})
});
}
fn process(socket: TcpStream) {
let (tx, rx) =
// Frame the socket using the `Http` protocol. This maps the TCP socket
// to a Stream + Sink of HTTP frames.
Http.framed(socket)
// This splits a single `Stream + Sink` value into two separate handles
// that can be used independently (even on different tasks or threads).
.split();
// Map all requests into responses and send them back to the client.
let task = tx.send_all(rx.and_then(respond))
.then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
while let Some(Ok(stream)) = incoming.next().await {
tokio::spawn(async move {
if let Err(e) = process(stream).await {
println!("failed to process connection; error = {}", e);
}
Ok(())
});
}
// Spawn the task that handles the connection.
tokio::spawn(task);
Ok(())
}
/// "Server logic" is implemented in this function.
///
/// This function is a map from and HTTP request to a future of a response and
/// represents the various handling a server might do. Currently the contents
/// here are pretty uninteresting.
fn respond(req: Request<()>)
-> Box<Future<Item = Response<String>, Error = io::Error> + Send>
{
let mut ret = Response::builder();
async fn process(stream: TcpStream) -> Result<(), Box<dyn Error>> {
let mut transport = Framed::new(stream, Http);
while let Some(request) = transport.next().await {
match request {
Ok(request) => {
let response = respond(request).await?;
transport.send(response).await?;
}
Err(e) => return Err(e.into()),
}
}
Ok(())
}
async fn respond(req: Request<()>) -> Result<Response<String>, Box<dyn Error>> {
let mut response = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
ret.header("Content-Type", "text/plain");
response = response.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
"/json" => {
ret.header("Content-Type", "application/json");
response = response.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
}
serde_json::to_string(&Message { message: "Hello, World!" })
.unwrap()
serde_json::to_string(&Message {
message: "Hello, World!",
})?
}
_ => {
ret.status(StatusCode::NOT_FOUND);
response = response.status(StatusCode::NOT_FOUND);
String::new()
}
};
Box::new(future::ok(ret.body(body).unwrap()))
let response = response
.body(body)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Ok(response)
}
struct Http;
@@ -121,12 +103,19 @@ impl Encoder for Http {
fn encode(&mut self, item: Response<String>, dst: &mut BytesMut) -> io::Result<()> {
use std::fmt::Write;
write!(BytesWrite(dst), "\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
", item.status(), item.body().len(), date::now()).unwrap();
write!(
BytesWrite(dst),
"\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
",
item.status(),
item.body().len(),
date::now()
)
.unwrap();
for (k, v) in item.headers() {
dst.extend_from_slice(k.as_str().as_bytes());
@@ -145,13 +134,13 @@ impl Encoder for Http {
// doesn't go through io::Error.
struct BytesWrite<'a>(&'a mut BytesMut);
impl<'a> fmt::Write for BytesWrite<'a> {
impl fmt::Write for BytesWrite<'_> {
fn write_str(&mut self, s: &str) -> fmt::Result {
self.0.extend_from_slice(s.as_bytes());
Ok(())
}
fn write_fmt(&mut self, args: fmt::Arguments) -> fmt::Result {
fn write_fmt(&mut self, args: fmt::Arguments<'_>) -> fmt::Result {
fmt::write(self, args)
}
}
@@ -195,33 +184,39 @@ impl Decoder for Http {
headers[i] = Some((k, v));
}
(toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt)
(
toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt,
)
};
if version != 1 {
return Err(io::Error::new(io::ErrorKind::Other, "only HTTP/1.1 accepted"))
return Err(io::Error::new(
io::ErrorKind::Other,
"only HTTP/1.1 accepted",
));
}
let data = src.split_to(amt).freeze();
let mut ret = Request::builder();
ret.method(&data[method.0..method.1]);
ret.uri(data.slice(path.0, path.1));
ret.version(http::Version::HTTP_11);
ret = ret.method(&data[method.0..method.1]);
let s = data.slice(path.0..path.1);
let s = unsafe { String::from_utf8_unchecked(Vec::from(s.as_ref())) };
ret = ret.uri(s);
ret = ret.version(http::Version::HTTP_11);
for header in headers.iter() {
let (k, v) = match *header {
Some((ref k, ref v)) => (k, v),
None => break,
};
let value = unsafe {
HeaderValue::from_shared_unchecked(data.slice(v.0, v.1))
};
ret.header(&data[k.0..k.1], value);
let value = HeaderValue::from_bytes(data.slice(v.0..v.1).as_ref())
.map_err(|_| io::Error::new(io::ErrorKind::Other, "header decode error"))?;
ret = ret.header(&data[k.0..k.1], value);
}
let req = ret.body(()).map_err(|e| {
io::Error::new(io::ErrorKind::Other, e)
})?;
let req = ret
.body(())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
Ok(Some(req))
}
}
@@ -271,7 +266,7 @@ mod date {
}));
impl fmt::Display for Now {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
LAST.with(|cache| {
let mut cache = cache.borrow_mut();
let now = time::get_time();
@@ -298,7 +293,7 @@ mod date {
struct LocalBuffer<'a>(&'a mut LastRenderedNow);
impl<'a> fmt::Write for LocalBuffer<'a> {
impl fmt::Write for LocalBuffer<'_> {
fn write_str(&mut self, s: &str) -> fmt::Result {
let start = self.0.amt;
let end = start + s.len();
+27 -29
View File
@@ -26,49 +26,47 @@
//! Please mind that since the UDP protocol doesn't have any capabilities to detect a broken
//! connection the server needs to be run first, otherwise the client will block forever.
extern crate futures;
extern crate tokio;
#![warn(rust_2018_idioms)]
use std::env;
use std::io::stdin;
use std::error::Error;
use std::io::{stdin, Read};
use std::net::SocketAddr;
use tokio::net::UdpSocket;
use tokio::prelude::*;
fn get_stdin_data() -> Vec<u8> {
fn get_stdin_data() -> Result<Vec<u8>, Box<dyn std::error::Error>> {
let mut buf = Vec::new();
stdin().read_to_end(&mut buf).unwrap();
buf
stdin().read_to_end(&mut buf)?;
Ok(buf)
}
fn main() {
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let remote_addr: SocketAddr = env::args()
.nth(1)
.unwrap_or("127.0.0.1:8080".into())
.parse()
.unwrap();
.unwrap_or_else(|| "127.0.0.1:8080".into())
.parse()?;
// We use port 0 to let the operating system allocate an available port for us.
let local_addr: SocketAddr = if remote_addr.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
}.parse()
.unwrap();
let socket = UdpSocket::bind(&local_addr).unwrap();
const MAX_DATAGRAM_SIZE: usize = 65_507;
let processing = socket
.send_dgram(get_stdin_data(), &remote_addr)
.and_then(|(socket, _)| socket.recv_dgram(vec![0u8; MAX_DATAGRAM_SIZE]))
.map(|(_, data, len, _)| {
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
)
})
.wait();
match processing {
Ok(_) => {}
Err(e) => eprintln!("Encountered an error: {}", e),
}
.parse()?;
let mut socket = UdpSocket::bind(local_addr).await?;
const MAX_DATAGRAM_SIZE: usize = 65_507;
socket.connect(&remote_addr).await?;
let data = get_stdin_data()?;
socket.send(&data).await?;
let mut data = vec![0u8; MAX_DATAGRAM_SIZE];
let len = socket.recv(&mut data).await?;
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
);
Ok(())
}
+58 -42
View File
@@ -1,64 +1,80 @@
//! This example leverages `BytesCodec` to create a UDP client and server which
//! speak a custom protocol.
//!
//! Here we're using the codec from tokio-io to convert a UDP socket to a stream of
//! Here we're using the codec from `tokio-codec` to convert a UDP socket to a stream of
//! client messages. These messages are then processed and returned back as a
//! new message with a new destination. Overall, we then use this to construct a
//! "ping pong" pair where two sockets are sending messages back and forth.
#![deny(warnings)]
#![warn(rust_2018_idioms)]
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate env_logger;
use tokio::net::UdpSocket;
use tokio::stream::StreamExt;
use tokio::{io, time};
use tokio_util::codec::BytesCodec;
use tokio_util::udp::UdpFramed;
use bytes::Bytes;
use futures::{FutureExt, SinkExt};
use std::env;
use std::error::Error;
use std::net::SocketAddr;
use std::time::Duration;
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_codec::BytesCodec;
fn main() {
let _ = env_logger::init();
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:0".to_string());
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr).unwrap();
let b = UdpSocket::bind(&addr).unwrap();
let b_addr = b.local_addr().unwrap();
let a = UdpSocket::bind(&addr).await?;
let b = UdpSocket::bind(&addr).await?;
// We're parsing each socket with the `BytesCodec` included in `tokio_io`, and then we
// `split` each codec into the sink/stream halves.
let (a_sink, a_stream) = UdpFramed::new(a, BytesCodec::new()).split();
let (b_sink, b_stream) = UdpFramed::new(b, BytesCodec::new()).split();
let b_addr = b.local_addr()?;
let mut a = UdpFramed::new(a, BytesCodec::new());
let mut b = UdpFramed::new(b, BytesCodec::new());
// Start off by sending a ping from a to b, afterwards we just print out
// what they send us and continually send pings
// let pings = stream::iter((0..5).map(Ok));
let a = a_sink.send(("PING".into(), b_addr)).and_then(|a_sink| {
let mut i = 0;
let a_stream = a_stream.take(4).map(move |(msg, addr)| {
i += 1;
println!("[a] recv: {}", String::from_utf8_lossy(&msg));
(format!("PING {}", i).into(), addr)
});
a_sink.send_all(a_stream)
});
let a = ping(&mut a, b_addr);
// The second client we have will receive the pings from `a` and then send
// back pongs.
let b_stream = b_stream.map(|(msg, addr)| {
println!("[b] recv: {}", String::from_utf8_lossy(&msg));
("PONG".into(), addr)
});
let b = b_sink.send_all(b_stream);
let b = pong(&mut b);
// Spawn the sender of pongs and then wait for our pinger to finish.
tokio::run({
b.join(a)
.map(|_| ())
.map_err(|e| println!("error = {:?}", e))
});
// Run both futures simultaneously of `a` and `b` sending messages back and forth.
match futures::future::try_join(a, b).await {
Err(e) => println!("an error occured; error = {:?}", e),
_ => println!("done!"),
}
Ok(())
}
async fn ping(socket: &mut UdpFramed<BytesCodec>, b_addr: SocketAddr) -> Result<(), io::Error> {
socket.send((Bytes::from(&b"PING"[..]), b_addr)).await?;
for _ in 0..4usize {
let (bytes, addr) = socket.next().map(|e| e.unwrap()).await?;
println!("[a] recv: {}", String::from_utf8_lossy(&bytes));
socket.send((Bytes::from(&b"PING"[..]), addr)).await?;
}
Ok(())
}
async fn pong(socket: &mut UdpFramed<BytesCodec>) -> Result<(), io::Error> {
let timeout = Duration::from_millis(200);
while let Ok(Some(Ok((bytes, addr)))) = time::timeout(timeout, socket.next()).await {
println!("[b] recv: {}", String::from_utf8_lossy(&bytes));
socket.send((Bytes::from(&b"PONG"[..]), addr)).await?;
}
Ok(())
}
+1
View File
@@ -0,0 +1 @@
edition = "2018"
-15
View File
@@ -1,15 +0,0 @@
//! A configurable source of time.
//!
//! This module provides the [`now`][n] function, which returns an `Instant`
//! representing "now". The source of time used by this function is configurable
//! (via the [`tokio-timer`] crate) and allows mocking out the source of time in
//! tests or performing caching operations to reduce the number of syscalls.
//!
//! Note that, because the source of time is configurable, it is possible to
//! observe non-monotonic behavior when calling [`now`] from different
//! executors.
//!
//! [n]: fn.now.html
//! [`tokio-timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/clock/index.html
pub use tokio_timer::clock::now;
-835
View File
@@ -1,835 +0,0 @@
//! Execute many tasks concurrently on the current thread.
//!
//! [`CurrentThread`] is an executor that keeps tasks on the same thread that
//! they were spawned from. This allows it to execute futures that are not
//! `Send`.
//!
//! A single [`CurrentThread`] instance is able to efficiently manage a large
//! number of tasks and will attempt to schedule all tasks fairly.
//!
//! All tasks that are being managed by a [`CurrentThread`] executor are able to
//! spawn additional tasks by calling [`spawn`]. This function only works from
//! within the context of a running [`CurrentThread`] instance.
//!
//! The easiest way to start a new [`CurrentThread`] executor is to call
//! [`block_on_all`] with an initial task to seed the executor.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread;
//! use futures::future::lazy;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! current_thread::block_on_all(lazy(|| {
//! // The execution context is setup, futures may be executed.
//! current_thread::spawn(lazy(|| {
//! println!("called from the current thread executor");
//! Ok(())
//! }));
//!
//! Ok::<_, ()>(())
//! }));
//! # }
//! ```
//!
//! The `block_on_all` function will block the current thread until **all**
//! tasks that have been spawned onto the [`CurrentThread`] instance have
//! completed.
//!
//! More fine-grain control can be achieved by using [`CurrentThread`] directly.
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::current_thread::CurrentThread;
//! use futures::future::{lazy, empty};
//! use std::time::Duration;
//!
//! // Calling execute here results in a panic
//! // current_thread::spawn(my_future);
//!
//! # pub fn main() {
//! let mut current_thread = CurrentThread::new();
//!
//! // Spawn a task, the task is not executed yet.
//! current_thread.spawn(lazy(|| {
//! println!("Spawning a task");
//! Ok(())
//! }));
//!
//! // Spawn a task that never completes
//! current_thread.spawn(empty());
//!
//! // Run the executor, but only until the provided future completes. This
//! // provides the opportunity to start executing previously spawned tasks.
//! let res = current_thread.block_on(lazy(|| {
//! Ok::<_, ()>("Hello")
//! })).unwrap();
//!
//! // Now, run the executor for *at most* 1 second. Since a task was spawned
//! // that never completes, this function will return with an error.
//! current_thread.run_timeout(Duration::from_secs(1)).unwrap_err();
//! # }
//! ```
//!
//! # Execution model
//!
//! Internally, [`CurrentThread`] maintains a queue. When one of its tasks is
//! notified, the task gets added to the queue. The executor will pop tasks from
//! the queue and call [`Future::poll`]. If the task gets notified while it is
//! being executed, it won't get re-executed until all other tasks currently in
//! the queue get polled.
//!
//! Before the task is polled, a thread-local variable referencing the current
//! [`CurrentThread`] instance is set. This enables [`spawn`] to spawn new tasks
//! onto the same executor without having to thread through a handle value.
//!
//! If the [`CurrentThread`] instance still has uncompleted tasks, but none of
//! these tasks are ready to be polled, the current thread is put to sleep. When
//! a task is notified, the thread is woken up and processing resumes.
//!
//! All tasks managed by [`CurrentThread`] remain on the current thread. When a
//! task completes, it is dropped.
//!
//! [`spawn`]: fn.spawn.html
//! [`block_on_all`]: fn.block_on_all.html
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
#![allow(deprecated)]
mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::{self, Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::{executor, Async, Future};
use futures::future::{self, Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::marker::PhantomData;
use std::rc::Rc;
use std::time::{Duration, Instant};
use std::sync::mpsc;
#[cfg(feature = "unstable-futures")]
use futures2;
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
/// Execute futures and receive unpark notifications.
scheduler: Scheduler<P::Unpark>,
/// Current number of futures being executed
num_futures: usize,
/// Thread park handle
park: P,
/// Handle for spawning new futures from other threads
spawn_handle: Handle,
/// Receiver for futures spawned from other threads
spawn_receiver: mpsc::Receiver<Box<Future<Item = (), Error = ()> + Send + 'static>>,
}
/// Executes futures on the current thread.
///
/// All futures executed using this executor will be executed on the current
/// thread. As such, `run` will wait for these futures to complete before
/// returning.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
// Prevent the handle from moving across threads.
_p: ::std::marker::PhantomData<Rc<()>>,
}
/// Returned by the `turn` function.
#[derive(Debug)]
pub struct Turn {
polled: bool
}
impl Turn {
/// `true` if any futures were polled at all and `false` otherwise.
pub fn has_polled(&self) -> bool {
self.polled
}
}
/// A `CurrentThread` instance bound to a supplied execution context.
pub struct Entered<'a, P: Park + 'a> {
executor: &'a mut CurrentThread<P>,
enter: &'a mut Enter,
}
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[derive(Debug)]
pub struct Context<'a> {
cancel: Cell<bool>,
_p: PhantomData<&'a ()>,
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
_p: (),
}
/// Error returned by the `run_timeout` function.
#[derive(Debug)]
pub struct RunTimeoutError {
timeout: bool,
}
/// Error returned by the `turn` function.
#[derive(Debug)]
pub struct TurnError {
_p: (),
}
/// Error returned by the `block_on` function.
#[derive(Debug)]
pub struct BlockError<T> {
inner: Option<T>,
}
/// This is mostly split out to make the borrow checker happy.
struct Borrow<'a, U: 'a> {
scheduler: &'a mut Scheduler<U>,
num_futures: &'a mut usize,
}
trait SpawnLocal {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>);
}
struct CurrentRunner {
spawn: Cell<Option<*mut SpawnLocal>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
});
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[allow(deprecated)]
pub fn run<F, R>(f: F) -> R
where F: FnOnce(&mut Context) -> R
{
let mut context = Context {
cancel: Cell::new(false),
_p: PhantomData,
};
let mut current_thread = CurrentThread::new();
let ret = current_thread
.block_on(future::lazy(|| Ok::<_, ()>(f(&mut context))))
.unwrap();
if context.cancel.get() {
return ret;
}
current_thread.run().unwrap();
ret
}
/// Run the executor bootstrapping the execution with the provided future.
///
/// This creates a new [`CurrentThread`] executor, spawns the provided future,
/// and blocks the current thread until the provided future and **all**
/// subsequently spawned futures complete. In other words:
///
/// * If the provided bootstrap future does **not** spawn any additional tasks,
/// `block_on_all` returns once `future` completes.
/// * If the provided bootstrap future **does** spawn additional tasks, then
/// `block_on_all` returns once **all** spawned futures complete.
///
/// See [module level][mod] documentation for more details.
///
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
{
let mut current_thread = CurrentThread::new();
let ret = current_thread.block_on(future);
current_thread.run().unwrap();
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
}
/// Executes a future on the current thread.
///
/// The provided future must complete or be canceled before `run` will return.
///
/// Unlike [`tokio::spawn`], this function will always spawn on a
/// `CurrentThread` executor and is able to spawn futures that are not `Send`.
///
/// # Panics
///
/// This function can only be invoked from the context of a `run` call; any
/// other use will result in a panic.
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
{
TaskExecutor::current()
.spawn_local(Box::new(future))
.unwrap();
}
// ===== impl CurrentThread =====
impl CurrentThread<ParkThread> {
/// Create a new instance of `CurrentThread`.
pub fn new() -> Self {
CurrentThread::new_with_park(ParkThread::new())
}
}
impl<P: Park> CurrentThread<P> {
/// Create a new instance of `CurrentThread` backed by the given park
/// handle.
pub fn new_with_park(park: P) -> Self {
let unpark = park.unpark();
let (spawn_sender, spawn_receiver) = mpsc::channel();
let scheduler = Scheduler::new(unpark);
let notify = scheduler.notify();
CurrentThread {
scheduler: scheduler,
num_futures: 0,
park,
spawn_handle: Handle { sender: spawn_sender, notify: notify },
spawn_receiver: spawn_receiver,
}
}
/// Returns `true` if the executor is currently idle.
///
/// An idle executor is defined by not currently having any spawned tasks.
pub fn is_idle(&self) -> bool {
self.num_futures == 0
}
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).turn(duration)
}
/// Bind `CurrentThread` instance with an execution context.
pub fn enter<'a>(&'a mut self, enter: &'a mut Enter) -> Entered<'a, P> {
Entered {
executor: self,
enter,
}
}
/// Returns a reference to the underlying `Park` instance.
pub fn get_park(&self) -> &P {
&self.park
}
/// Returns a mutable reference to the underlying `Park` instance.
pub fn get_park_mut(&mut self) -> &mut P {
&mut self.park
}
fn borrow(&mut self) -> Borrow<P::Unpark> {
Borrow {
scheduler: &mut self.scheduler,
num_futures: &mut self.num_futures,
}
}
/// Get a new handle to spawn futures on the executor
///
/// Different to the executor itself, the handle can be sent to different
/// threads and can be used to spawn futures on the executor.
pub fn handle(&self) -> Handle {
self.spawn_handle.clone()
}
}
impl tokio_executor::Executor for CurrentThread {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.borrow().spawn_local(future);
Ok(())
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, _future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
panic!("Futures 0.2 integration is not available for current_thread");
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures)
.finish()
}
}
// ===== impl Entered =====
impl<'a, P: Park> Entered<'a, P> {
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
match res {
Ok(Async::Ready(e)) => return Ok(e),
Err(e) => return Err(BlockError { inner: Some(e) }),
Ok(Async::NotReady) => {}
}
self.tick();
if let Err(_) = self.executor.park.park() {
return Err(BlockError { inner: None });
}
}
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let res = if self.executor.scheduler.has_pending_futures() {
self.executor.park.park_timeout(Duration::from_millis(0))
} else {
match duration {
Some(duration) => self.executor.park.park_timeout(duration),
None => self.executor.park.park(),
}
};
if res.is_err() {
return Err(TurnError { _p: () });
}
let polled = self.tick();
Ok(Turn { polled })
}
/// Returns a reference to the underlying `Park` instance.
pub fn get_park(&self) -> &P {
&self.executor.park
}
/// Returns a mutable reference to the underlying `Park` instance.
pub fn get_park_mut(&mut self) -> &mut P {
&mut self.executor.park
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
if self.executor.is_idle() {
// Nothing to do
return Ok(());
}
let mut time = dur.map(|dur| (Instant::now() + dur, dur));
loop {
self.tick();
if self.executor.is_idle() {
return Ok(());
}
match time {
Some((until, rem)) => {
if let Err(_) = self.executor.park.park_timeout(rem) {
return Err(RunTimeoutError::new(false));
}
let now = Instant::now();
if now >= until {
return Err(RunTimeoutError::new(true));
}
time = Some((until, until - now));
}
None => {
if let Err(_) = self.executor.park.park() {
return Err(RunTimeoutError::new(false));
}
}
}
}
}
/// Returns `true` if any futures were processed
fn tick(&mut self) -> bool {
// Spawn any futures that were spawned from other threads by manually
// looping over the receiver stream
// FIXME: Slightly ugly but needed to make the borrow checker happy
let (mut borrow, spawn_receiver) = (
Borrow {
scheduler: &mut self.executor.scheduler,
num_futures: &mut self.executor.num_futures,
},
&mut self.executor.spawn_receiver,
);
while let Ok(future) = spawn_receiver.try_recv() {
borrow.spawn_local(future);
}
// After any pending futures were scheduled, do the actual tick
borrow.scheduler.tick(
&mut *self.enter,
borrow.num_futures)
}
}
impl<'a, P: Park> fmt::Debug for Entered<'a, P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Entered")
.field("executor", &self.executor)
.field("enter", &self.enter)
.finish()
}
}
// ===== impl Handle =====
/// Handle to spawn a future on the corresponding `CurrentThread` instance
#[derive(Clone)]
pub struct Handle {
sender: mpsc::Sender<Box<Future<Item = (), Error = ()> + Send + 'static>>,
notify: executor::NotifyHandle,
}
// Manual implementation because the Sender does not implement Debug
impl fmt::Debug for Handle {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Handle")
.finish()
}
}
impl Handle {
/// Spawn a future onto the `CurrentThread` instance corresponding to this handle
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the `CurrentThread`
/// instance of the `Handle` does not exist anymore.
pub fn spawn<F>(&self, future: F) -> Result<(), SpawnError>
where F: Future<Item = (), Error = ()> + Send + 'static {
self.sender.send(Box::new(future))
.expect("CurrentThread does not exist anymore");
// use 0 for the id, CurrentThread does not make use of it
self.notify.notify(0);
Ok(())
}
}
// ===== impl TaskExecutor =====
#[deprecated(since = "0.1.2", note = "use TaskExecutor::current instead")]
#[doc(hidden)]
pub fn task_executor() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
impl TaskExecutor {
/// Returns an executor that executes futures on the current thread.
///
/// The user of `TaskExecutor` must ensure that when a future is submitted,
/// that it is done within the context of a call to `run`.
///
/// For more details, see the [module level](index.html) documentation.
pub fn current() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
}
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.spawn_local(future)
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, _future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
panic!("Futures 0.2 integration is not available for current_thread");
}
fn status(&self) -> Result<(), SpawnError> {
CURRENT.with(|current| {
if current.spawn.get().is_some() {
Ok(())
} else {
Err(SpawnError::shutdown())
}
})
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future)) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
}
})
}
}
// ===== impl Context =====
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
// ===== impl Borrow =====
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.set_spawn(self, || {
f()
})
})
}
}
impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>) {
*self.num_futures += 1;
self.scheduler.schedule(future);
}
}
// ===== impl CurrentRunner =====
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
{
struct Reset<'a>(&'a CurrentRunner);
impl<'a> Drop for Reset<'a> {
fn drop(&mut self) {
self.0.spawn.set(None);
}
}
let _reset = Reset(self);
let spawn = unsafe { hide_lt(spawn as *mut SpawnLocal) };
self.spawn.set(Some(spawn));
f()
}
}
unsafe fn hide_lt<'a>(p: *mut (SpawnLocal + 'a)) -> *mut (SpawnLocal + 'static) {
use std::mem;
mem::transmute(p)
}
// ===== impl RunTimeoutError =====
impl RunTimeoutError {
fn new(timeout: bool) -> Self {
RunTimeoutError { timeout }
}
/// Returns `true` if the error was caused by the operation timing out.
pub fn is_timeout(&self) -> bool {
self.timeout
}
}
impl From<tokio_executor::EnterError> for RunTimeoutError {
fn from(_: tokio_executor::EnterError) -> Self {
RunTimeoutError::new(false)
}
}
// ===== impl BlockError =====
impl<T> BlockError<T> {
/// Returns the error yielded by the future being blocked on
pub fn into_inner(self) -> Option<T> {
self.inner
}
}
impl<T> From<tokio_executor::EnterError> for BlockError<T> {
fn from(_: tokio_executor::EnterError) -> Self {
BlockError { inner: None }
}
}
-772
View File
@@ -1,772 +0,0 @@
use super::Borrow;
use tokio_executor::Enter;
use tokio_executor::park::Unpark;
use futures::{Future, Async};
use futures::executor::{self, Spawn, UnsafeNotify, NotifyHandle};
use std::cell::UnsafeCell;
use std::fmt::{self, Debug};
use std::mem;
use std::ptr;
use std::sync::atomic::Ordering::{Relaxed, SeqCst, Acquire, Release, AcqRel};
use std::sync::atomic::{AtomicPtr, AtomicBool, AtomicUsize};
use std::sync::{Arc, Weak};
use std::usize;
use std::thread;
use std::marker::PhantomData;
/// A generic task-aware scheduler.
///
/// This is used both by `FuturesUnordered` and the current-thread executor.
pub struct Scheduler<U> {
inner: Arc<Inner<U>>,
nodes: List<U>,
}
pub struct Notify<'a, U: 'a>(&'a Arc<Node<U>>);
// A linked-list of nodes
struct List<U> {
len: usize,
head: *const Node<U>,
tail: *const Node<U>,
}
// Scheduler is implemented using two linked lists. The first linked list tracks
// all items managed by a `Scheduler`. This list is stored on the `Scheduler`
// struct and is **not** thread safe. The second linked list is an
// implementation of the intrusive MPSC queue algorithm described by
// 1024cores.net and is stored on `Inner`. This linked list can push items to
// the back concurrently but only one consumer may pop from the front. To
// enforce this requirement, all popping will be performed via fns on
// `Scheduler` that take `&mut self`.
//
// When a item is submitted to the set a node is allocated and inserted in
// both linked lists. This means that all insertion operations **must** be
// originated from `Scheduler` with `&mut self` The next call to `tick` will
// (eventually) see this node and call `poll` on the item.
//
// Nodes are wrapped in `Arc` cells which manage the lifetime of the node.
// However, `Arc` handles are sometimes cast to `*const Node` pointers.
// Specifically, when a node is stored in at least one of the two lists
// described above, this represents a logical `Arc` handle. This is how
// `Scheduler` maintains its reference to all nodes it manages. Each
// `NotifyHandle` instance is an `Arc<Node>` as well.
//
// When `Scheduler` drops, it clears the linked list of all nodes that it
// manages. When doing so, it must attempt to decrement the reference count (by
// dropping an Arc handle). However, it can **only** decrement the reference
// count if the node is not currently stored in the mpsc channel. If the node
// **is** "queued" in the mpsc channel, then the arc reference count cannot be
// decremented. Once the node is popped from the mpsc channel, then the final
// arc reference count can be decremented, thus freeing the node.
struct Inner<U> {
// Thread unpark handle
unpark: U,
// Tick number
tick_num: AtomicUsize,
// Head/tail of the readiness queue
head_readiness: AtomicPtr<Node<U>>,
tail_readiness: UnsafeCell<*const Node<U>>,
// Used as part of the MPSC queue algorithm
stub: Arc<Node<U>>,
}
unsafe impl<U: Sync + Send> Send for Inner<U> {}
unsafe impl<U: Sync + Send> Sync for Inner<U> {}
impl<U: Unpark> executor::Notify for Inner<U> {
fn notify(&self, _: usize) {
self.unpark.unpark();
}
}
struct Node<U> {
// The item
item: UnsafeCell<Option<Task>>,
// The tick at which this node was notified
notified_at: AtomicUsize,
// Next pointer for linked list tracking all active nodes
next_all: UnsafeCell<*const Node<U>>,
// Previous node in linked list tracking all active nodes
prev_all: UnsafeCell<*const Node<U>>,
// Next pointer in readiness queue
next_readiness: AtomicPtr<Node<U>>,
// Whether or not this node is currently in the mpsc queue.
queued: AtomicBool,
// Queue that we'll be enqueued to when notified
queue: Weak<Inner<U>>,
}
/// Returned by `Inner::dequeue`, representing either a dequeue success (with
/// the dequeued node), an empty list, or an inconsistent state.
///
/// The inconsistent state is described in more detail at [1024cores], but
/// roughly indicates that a node will be ready to dequeue sometime shortly in
/// the future and the caller should try again soon.
///
/// [1024cores]: http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue
enum Dequeue<U> {
Data(*const Node<U>),
Empty,
Yield,
Inconsistent,
}
/// Wraps a spawned boxed future
struct Task(Spawn<Box<Future<Item = (), Error = ()>>>);
/// A task that is scheduled. `turn` must be called
pub struct Scheduled<'a, U: 'a> {
task: &'a mut Task,
notify: &'a Notify<'a, U>,
done: &'a mut bool,
}
impl<U> Scheduler<U>
where U: Unpark,
{
/// Constructs a new, empty `Scheduler`
///
/// The returned `Scheduler` does not contain any items and, in this
/// state, `Scheduler::poll` will return `Ok(Async::Ready(None))`.
pub fn new(unpark: U) -> Self {
let stub = Arc::new(Node {
item: UnsafeCell::new(None),
notified_at: AtomicUsize::new(0),
next_all: UnsafeCell::new(ptr::null()),
prev_all: UnsafeCell::new(ptr::null()),
next_readiness: AtomicPtr::new(ptr::null_mut()),
queued: AtomicBool::new(true),
queue: Weak::new(),
});
let stub_ptr = &*stub as *const Node<U>;
let inner = Arc::new(Inner {
unpark,
tick_num: AtomicUsize::new(0),
head_readiness: AtomicPtr::new(stub_ptr as *mut _),
tail_readiness: UnsafeCell::new(stub_ptr),
stub: stub,
});
Scheduler {
inner: inner,
nodes: List::new(),
}
}
pub fn notify(&self) -> NotifyHandle {
self.inner.clone().into()
}
pub fn schedule(&mut self, item: Box<Future<Item = (), Error = ()>>) {
// Get the current scheduler tick
let tick_num = self.inner.tick_num.load(SeqCst);
let node = Arc::new(Node {
item: UnsafeCell::new(Some(Task::new(item))),
notified_at: AtomicUsize::new(tick_num),
next_all: UnsafeCell::new(ptr::null_mut()),
prev_all: UnsafeCell::new(ptr::null_mut()),
next_readiness: AtomicPtr::new(ptr::null_mut()),
queued: AtomicBool::new(true),
queue: Arc::downgrade(&self.inner),
});
// Right now our node has a strong reference count of 1. We transfer
// ownership of this reference count to our internal linked list
// and we'll reclaim ownership through the `unlink` function below.
let ptr = self.nodes.push_back(node);
// We'll need to get the item "into the system" to start tracking it,
// e.g. getting its unpark notifications going to us tracking which
// items are ready. To do that we unconditionally enqueue it for
// polling here.
self.inner.enqueue(ptr);
}
/// Returns `true` if there are currently any pending futures
pub fn has_pending_futures(&mut self) -> bool {
// See function definition for why the unsafe is needed and
// correctly used here
unsafe {
self.inner.has_pending_futures()
}
}
/// Advance the scheduler state, returning `true` if any futures were
/// processed.
///
/// This function should be called whenever the caller is notified via a
/// wakeup.
pub fn tick(&mut self, enter: &mut Enter, num_futures: &mut usize) -> bool
{
let mut ret = false;
let tick = self.inner.tick_num.fetch_add(1, SeqCst)
.wrapping_add(1);
loop {
let node = match unsafe { self.inner.dequeue(Some(tick)) } {
Dequeue::Empty => {
return ret;
}
Dequeue::Yield => {
self.inner.unpark.unpark();
return ret;
}
Dequeue::Inconsistent => {
thread::yield_now();
continue;
}
Dequeue::Data(node) => node,
};
ret = true;
debug_assert!(node != self.inner.stub());
unsafe {
if (*(*node).item.get()).is_none() {
// The node has already been released. However, while it was
// being released, another thread notified it, which
// resulted in it getting pushed into the mpsc channel.
//
// In this case, we just dec the ref count.
let node = ptr2arc(node);
assert!((*node.next_all.get()).is_null());
assert!((*node.prev_all.get()).is_null());
continue
};
// We're going to need to be very careful if the `poll`
// function below panics. We need to (a) not leak memory and
// (b) ensure that we still don't have any use-after-frees. To
// manage this we do a few things:
//
// * This "bomb" here will call `release_node` if dropped
// abnormally. That way we'll be sure the memory management
// of the `node` is managed correctly.
//
// * We unlink the node from our internal queue to preemptively
// assume is is complete (will return Ready or panic), in
// which case we'll want to discard it regardless.
//
struct Bomb<'a, U: Unpark + 'a> {
borrow: &'a mut Borrow<'a, U>,
enter: &'a mut Enter,
node: Option<Arc<Node<U>>>,
}
impl<'a, U: Unpark> Drop for Bomb<'a, U> {
fn drop(&mut self) {
if let Some(node) = self.node.take() {
self.borrow.enter(self.enter, || release_node(node))
}
}
}
let node = self.nodes.remove(node);
let mut borrow = Borrow {
scheduler: self,
num_futures,
};
let mut bomb = Bomb {
node: Some(node),
enter: enter,
borrow: &mut borrow,
};
let mut done = false;
// Now that the bomb holds the node, create a new scope. This
// scope ensures that the borrow will go out of scope before we
// mutate the node pointer in `bomb` again
{
let node = bomb.node.as_ref().unwrap();
// Get a reference to the inner future. We already ensured
// that the item `is_some`.
let item = (*node.item.get()).as_mut().unwrap();
// Unset queued flag... this must be done before
// polling. This ensures that the item gets
// rescheduled if it is notified **during** a call
// to `poll`.
let prev = (*node).queued.swap(false, SeqCst);
assert!(prev);
// Poll the underlying item with the appropriate `notify`
// implementation. This is where a large bit of the unsafety
// starts to stem from internally. The `notify` instance itself
// is basically just our `Arc<Node>` and tracks the mpsc
// queue of ready items.
//
// Critically though `Node` won't actually access `Task`, the
// item, while it's floating around inside of `Task`
// instances. These structs will basically just use `T` to size
// the internal allocation, appropriately accessing fields and
// deallocating the node if need be.
let borrow = &mut *bomb.borrow;
let enter = &mut *bomb.enter;
let notify = Notify(bomb.node.as_ref().unwrap());
let mut scheduled = Scheduled {
task: item,
notify: &notify,
done: &mut done,
};
if borrow.enter(enter, || scheduled.tick()) {
*borrow.num_futures -= 1;
}
}
if !done {
// The future is not done, push it back into the "all
// node" list.
let node = bomb.node.take().unwrap();
bomb.borrow.scheduler.nodes.push_back(node);
}
}
}
}
}
impl<'a, U: Unpark> Scheduled<'a, U> {
/// Polls the task, returns `true` if the task has completed.
pub fn tick(&mut self) -> bool {
// Tick the future
let ret = match self.task.0.poll_future_notify(self.notify, 0) {
Ok(Async::Ready(_)) | Err(_) => true,
Ok(Async::NotReady) => false,
};
*self.done = ret;
ret
}
}
impl Task {
pub fn new(future: Box<Future<Item = (), Error = ()> + 'static>) -> Self {
Task(executor::spawn(future))
}
}
impl fmt::Debug for Task {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Task")
.finish()
}
}
fn release_node<U>(node: Arc<Node<U>>) {
// The item is done, try to reset the queued flag. This will prevent
// `notify` from doing any work in the item
let prev = node.queued.swap(true, SeqCst);
// Drop the item, even if it hasn't finished yet. This is safe
// because we're dropping the item on the thread that owns
// `Scheduler`, which correctly tracks T's lifetimes and such.
unsafe {
drop((*node.item.get()).take());
}
// If the queued flag was previously set then it means that this node
// is still in our internal mpsc queue. We then transfer ownership
// of our reference count to the mpsc queue, and it'll come along and
// free it later, noticing that the item is `None`.
//
// If, however, the queued flag was *not* set then we're safe to
// release our reference count on the internal node. The queued flag
// was set above so all item `enqueue` operations will not actually
// enqueue the node, so our node will never see the mpsc queue again.
// The node itself will be deallocated once all reference counts have
// been dropped by the various owning tasks elsewhere.
if prev {
mem::forget(node);
}
}
impl<U> Debug for Scheduler<U> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "Scheduler {{ ... }}")
}
}
impl<U> Drop for Scheduler<U> {
fn drop(&mut self) {
// When a `Scheduler` is dropped we want to drop all items associated
// with it. At the same time though there may be tons of `Task` handles
// flying around which contain `Node` references inside them. We'll
// let those naturally get deallocated when the `Task` itself goes out
// of scope or gets notified.
while let Some(node) = self.nodes.pop_front() {
release_node(node);
}
// Note that at this point we could still have a bunch of nodes in the
// mpsc queue. None of those nodes, however, have items associated
// with them so they're safe to destroy on any thread. At this point
// the `Scheduler` struct, the owner of the one strong reference
// to `Inner` will drop the strong reference. At that point
// whichever thread releases the strong refcount last (be it this
// thread or some other thread as part of an `upgrade`) will clear out
// the mpsc queue and free all remaining nodes.
//
// While that freeing operation isn't guaranteed to happen here, it's
// guaranteed to happen "promptly" as no more "blocking work" will
// happen while there's a strong refcount held.
}
}
impl<U> Inner<U> {
/// The enqueue function from the 1024cores intrusive MPSC queue algorithm.
fn enqueue(&self, node: *const Node<U>) {
unsafe {
debug_assert!((*node).queued.load(Relaxed));
// This action does not require any coordination
(*node).next_readiness.store(ptr::null_mut(), Relaxed);
// Note that these atomic orderings come from 1024cores
let node = node as *mut _;
let prev = self.head_readiness.swap(node, AcqRel);
(*prev).next_readiness.store(node, Release);
}
}
/// Returns `true` if there are currently any pending futures
///
/// See `dequeue` for an explanation why this function is unsafe.
unsafe fn has_pending_futures(&self) -> bool {
let tail = *self.tail_readiness.get();
let next = (*tail).next_readiness.load(Acquire);
if tail == self.stub() {
if next.is_null() {
return false;
}
}
true
}
/// The dequeue function from the 1024cores intrusive MPSC queue algorithm
///
/// Note that this unsafe as it required mutual exclusion (only one thread
/// can call this) to be guaranteed elsewhere.
unsafe fn dequeue(&self, tick: Option<usize>) -> Dequeue<U> {
let mut tail = *self.tail_readiness.get();
let mut next = (*tail).next_readiness.load(Acquire);
if tail == self.stub() {
if next.is_null() {
return Dequeue::Empty;
}
*self.tail_readiness.get() = next;
tail = next;
next = (*next).next_readiness.load(Acquire);
}
if let Some(tick) = tick {
let actual = (*tail).notified_at.load(SeqCst);
// Only dequeue if the node was not scheduled during the current
// tick.
if actual == tick {
// Only doing the check above **should** be enough in
// practice. However, technically there is a potential for
// deadlocking if there are `usize::MAX` ticks while the thread
// scheduling the task is frozen.
//
// If, for some reason, this is not enough, calling `unpark`
// here will resolve the issue.
return Dequeue::Yield;
}
}
if !next.is_null() {
*self.tail_readiness.get() = next;
debug_assert!(tail != self.stub());
return Dequeue::Data(tail);
}
if self.head_readiness.load(Acquire) as *const _ != tail {
return Dequeue::Inconsistent;
}
self.enqueue(self.stub());
next = (*tail).next_readiness.load(Acquire);
if !next.is_null() {
*self.tail_readiness.get() = next;
return Dequeue::Data(tail);
}
Dequeue::Inconsistent
}
fn stub(&self) -> *const Node<U> {
&*self.stub
}
}
impl<U> Drop for Inner<U> {
fn drop(&mut self) {
// Once we're in the destructor for `Inner` we need to clear out the
// mpsc queue of nodes if there's anything left in there.
//
// Note that each node has a strong reference count associated with it
// which is owned by the mpsc queue. All nodes should have had their
// items dropped already by the `Scheduler` destructor above,
// so we're just pulling out nodes and dropping their refcounts.
unsafe {
loop {
match self.dequeue(None) {
Dequeue::Empty => break,
Dequeue::Yield => unreachable!(),
Dequeue::Inconsistent => abort("inconsistent in drop"),
Dequeue::Data(ptr) => drop(ptr2arc(ptr)),
}
}
}
}
}
impl<U> List<U> {
fn new() -> Self {
List {
len: 0,
head: ptr::null_mut(),
tail: ptr::null_mut(),
}
}
/// Prepends an element to the back of the list
fn push_back(&mut self, node: Arc<Node<U>>) -> *const Node<U> {
let ptr = arc2ptr(node);
unsafe {
// Point to the current last node in the list
*(*ptr).prev_all.get() = self.tail;
*(*ptr).next_all.get() = ptr::null_mut();
if !self.tail.is_null() {
*(*self.tail).next_all.get() = ptr;
self.tail = ptr;
} else {
// This is the first node
self.tail = ptr;
self.head = ptr;
}
}
self.len += 1;
return ptr
}
/// Pop an element from the front of the list
fn pop_front(&mut self) -> Option<Arc<Node<U>>> {
if self.head.is_null() {
// The list is empty
return None;
}
self.len -= 1;
unsafe {
// Convert the ptr to Arc<_>
let node = ptr2arc(self.head);
// Update the head pointer
self.head = *node.next_all.get();
// If the pointer is null, then the list is empty
if self.head.is_null() {
self.tail = ptr::null_mut();
} else {
*(*self.head).prev_all.get() = ptr::null_mut();
}
Some(node)
}
}
/// Remove a specific node
unsafe fn remove(&mut self, node: *const Node<U>) -> Arc<Node<U>> {
let node = ptr2arc(node);
let next = *node.next_all.get();
let prev = *node.prev_all.get();
*node.next_all.get() = ptr::null_mut();
*node.prev_all.get() = ptr::null_mut();
if !next.is_null() {
*(*next).prev_all.get() = prev;
} else {
self.tail = prev;
}
if !prev.is_null() {
*(*prev).next_all.get() = next;
} else {
self.head = next;
}
self.len -= 1;
return node
}
}
impl<'a, U> Clone for Notify<'a, U> {
fn clone(&self) -> Self {
Notify(self.0)
}
}
impl<'a, U> fmt::Debug for Notify<'a, U> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Notify").finish()
}
}
impl<'a, U: Unpark> From<Notify<'a, U>> for NotifyHandle {
fn from(handle: Notify<'a, U>) -> NotifyHandle {
unsafe {
let ptr = handle.0.clone();
let ptr = mem::transmute::<Arc<Node<U>>, *mut ArcNode<U>>(ptr);
NotifyHandle::new(hide_lt(ptr))
}
}
}
struct ArcNode<U>(PhantomData<U>);
// We should never touch `Task` on any thread other than the one owning
// `Scheduler`, so this should be a safe operation.
unsafe impl<U: Sync + Send> Send for ArcNode<U> {}
unsafe impl<U: Sync + Send> Sync for ArcNode<U> {}
impl<U: Unpark> executor::Notify for ArcNode<U> {
fn notify(&self, _id: usize) {
unsafe {
let me: *const ArcNode<U> = self;
let me: *const *const ArcNode<U> = &me;
let me = me as *const Arc<Node<U>>;
Node::notify(&*me)
}
}
}
unsafe impl<U: Unpark> UnsafeNotify for ArcNode<U> {
unsafe fn clone_raw(&self) -> NotifyHandle {
let me: *const ArcNode<U> = self;
let me: *const *const ArcNode<U> = &me;
let me = &*(me as *const Arc<Node<U>>);
Notify(me).into()
}
unsafe fn drop_raw(&self) {
let mut me: *const ArcNode<U> = self;
let me = &mut me as *mut *const ArcNode<U> as *mut Arc<Node<U>>;
ptr::drop_in_place(me);
}
}
unsafe fn hide_lt<U: Unpark>(p: *mut ArcNode<U>) -> *mut UnsafeNotify {
mem::transmute(p as *mut UnsafeNotify)
}
impl<U: Unpark> Node<U> {
fn notify(me: &Arc<Node<U>>) {
let inner = match me.queue.upgrade() {
Some(inner) => inner,
None => return,
};
// It's our job to notify the node that it's ready to get polled,
// meaning that we need to enqueue it into the readiness queue. To
// do this we flag that we're ready to be queued, and if successful
// we then do the literal queueing operation, ensuring that we're
// only queued once.
//
// Once the node is inserted we be sure to notify the parent task,
// as it'll want to come along and pick up our node now.
//
// Note that we don't change the reference count of the node here,
// we're just enqueueing the raw pointer. The `Scheduler`
// implementation guarantees that if we set the `queued` flag true that
// there's a reference count held by the main `Scheduler` queue
// still.
let prev = me.queued.swap(true, SeqCst);
if !prev {
// Get the current scheduler tick
let tick_num = inner.tick_num.load(SeqCst);
me.notified_at.store(tick_num, SeqCst);
inner.enqueue(&**me);
inner.unpark.unpark();
}
}
}
impl<U> Drop for Node<U> {
fn drop(&mut self) {
// Currently a `Node` is sent across all threads for any lifetime,
// regardless of `T`. This means that for memory safety we can't
// actually touch `T` at any time except when we have a reference to the
// `Scheduler` itself.
//
// Consequently it *should* be the case that we always drop items from
// the `Scheduler` instance, but this is a bomb in place to catch
// any bugs in that logic.
unsafe {
if (*self.item.get()).is_some() {
abort("item still here when dropping");
}
}
}
}
fn arc2ptr<T>(ptr: Arc<T>) -> *const T {
let addr = &*ptr as *const T;
mem::forget(ptr);
return addr
}
unsafe fn ptr2arc<T>(ptr: *const T) -> Arc<T> {
let anchor = mem::transmute::<usize, Arc<T>>(0x10);
let addr = &*anchor as *const T;
mem::forget(anchor);
let offset = addr as isize - 0x10;
mem::transmute::<isize, Arc<T>>(ptr as isize - offset)
}
fn abort(s: &str) -> ! {
struct DoublePanic;
impl Drop for DoublePanic {
fn drop(&mut self) {
panic!("panicking twice to abort the program");
}
}
let _bomb = DoublePanic;
panic!("{}", s);
}
-239
View File
@@ -1,239 +0,0 @@
//! Task execution utilities.
//!
//! In the Tokio execution model, futures are lazy. When a future is created, no
//! work is performed. In order for the work defined by the future to happen,
//! the future must be submitted to an executor. A future that is submitted to
//! an executor is called a "task".
//!
//! The executor executor is responsible for ensuring that [`Future::poll`] is
//! called whenever the task is [notified]. Notification happens when the
//! internal state of a task transitions from "not ready" to ready. For
//! example, a socket might have received data and a call to `read` will now be
//! able to succeed.
//!
//! The specific strategy used to manage the tasks is left up to the
//! executor. There are two main flavors of executors: single-threaded and
//! multithreaded. This module provides both.
//!
//! * **[`current_thread`]**: A single-threaded executor that support spawning
//! tasks that are not `Send`. It guarantees that tasks will be executed on
//! the same thread from which they are spawned.
//!
//! * **[`thread_pool`]**: A multi-threaded executor that maintains a pool of
//! threads. Tasks are spawned to one of the threads in the pool and executed.
//! The pool employs a [work-stealing] strategy for optimizing how tasks get
//! spread across the available threads.
//!
//! # `Executor` trait.
//!
//! This module provides the [`Executor`] trait (re-exported from
//! [`tokio-executor`]), which describes the API that all executors must
//! implement.
//!
//! A free [`spawn`] function is provided that allows spawning futures onto the
//! default executor (tracked via a thread-local variable) without referencing a
//! handle. It is expected that all executors will set a value for the default
//! executor. This value will often be set to the executor itself, but it is
//! possible that the default executor might be set to a different executor.
//!
//! For example, the [`current_thread`] executor might set the default executor
//! to a thread pool instead of itself, allowing futures to spawn new tasks onto
//! the thread pool when those tasks are `Send`.
//!
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
//! [notified]: https://docs.rs/futures/0.1/futures/executor/trait.Notify.html#tymethod.notify
//! [`current_thread`]: current_thread/index.html
//! [`thread_pool`]: thread_pool/index.html
//! [work-stealing]: https://en.wikipedia.org/wiki/Work_stealing
//! [`tokio-executor`]: #
//! [`Executor`]: #
//! [`spawn`]: #
pub mod current_thread;
pub mod thread_pool {
//! Maintains a pool of threads across which the set of spawned tasks are
//! executed.
//!
//! [`ThreadPool`] is an executor that uses a thread pool for executing
//! tasks concurrently across multiple cores. It uses a thread pool that is
//! optimized for use cases that involve multiplexing large number of
//! independent tasks that perform short(ish) amounts of computation and are
//! mainly waiting on I/O, i.e. the Tokio use case.
//!
//! Usually, users of [`ThreadPool`] will not create pool instances.
//! Instead, they will create a [`Runtime`] instance, which comes with a
//! pre-configured thread pool.
//!
//! At the core, [`ThreadPool`] uses a work-stealing based scheduling
//! strategy. When spawning a task while *external* to the thread pool
//! (i.e., from a thread that is not part of the thread pool), the task is
//! randomly assigned to a worker thread. When spawning a task while
//! *internal* to the thread pool, the task is assigned to the current
//! worker.
//!
//! Each worker maintains its own queue and first focuses on processing all
//! tasks in its queue. When the worker's queue is empty, the worker will
//! attempt to *steal* tasks from other worker queues. This strategy helps
//! ensure that work is evenly distributed across threads while minimizing
//! synchronization between worker threads.
//!
//! # Usage
//!
//! Thread pool instances are created using [`ThreadPool::new`] or
//! [`Builder::new`]. The first option returns a thread pool with default
//! configuration values. The second option allows configuring the thread
//! pool before instantiating it.
//!
//! Once an instance is obtained, futures may be spawned onto it using the
//! [`spawn`] function.
//!
//! A handle to the thread pool is obtained using [`ThreadPool::sender`].
//! This handle is **only** able to spawn futures onto the thread pool. It
//! is unable to affect the lifecycle of the thread pool in any way. This
//! handle can be passed into functions or stored in structs as a way to
//! grant the capability of spawning futures.
//!
//! # Examples
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::thread_pool::ThreadPool;
//! use futures::future::{Future, lazy};
//!
//! # pub fn main() {
//! // Create a thread pool with default configuration values
//! let thread_pool = ThreadPool::new();
//!
//! thread_pool.spawn(lazy(|| {
//! println!("called from a worker thread");
//! Ok(())
//! }));
//!
//! // Gracefully shutdown the threadpool
//! thread_pool.shutdown().wait().unwrap();
//! # }
//! ```
//!
//! [`ThreadPool`]: struct.ThreadPool.html
//! [`ThreadPool::new`]: struct.ThreadPool.html#method.new
//! [`ThreadPool::sender`]: struct.ThreadPool.html#method.sender
//! [`spawn`]: struct.ThreadPool.html#method.spawn
//! [`Builder::new`]: struct.Builder.html#method.new
//! [`Runtime`]: ../../runtime/struct.Runtime.html
pub use tokio_threadpool::{
Builder,
Sender,
Shutdown,
ThreadPool,
};
}
pub use tokio_executor::{Executor, DefaultExecutor, SpawnError};
use futures::{Future, IntoFuture};
use futures::future::{self, FutureResult};
#[cfg(feature = "unstable-futures")]
use futures2;
/// Return value from the `spawn` function.
///
/// Currently this value doesn't actually provide any functionality. However, it
/// provides a way to add functionality later without breaking backwards
/// compatibility.
///
/// This also implements `IntoFuture` so that it can be used as the return value
/// in a `for_each` loop.
///
/// See [`spawn`] for more details.
///
/// [`spawn`]: fn.spawn.html
#[derive(Debug)]
pub struct Spawn(());
/// Spawns a future on the default executor.
///
/// In order for a future to do work, it must be spawned on an executor. The
/// `spawn` function is the easiest way to do this. It spawns a future on the
/// [default executor] for the current execution context (tracked using a
/// thread-local variable).
///
/// The default executor is **usually** a thread pool.
///
/// # Examples
///
/// In this example, a server is started and `spawn` is used to start a new task
/// that processes each received connection.
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// [default executor]: struct.DefaultExecutor.html
///
/// # Panics
///
/// This function will panic if the default executor is not set or if spawning
/// onto the default executor returns an error. To avoid the panic, use
/// [`DefaultExecutor`].
///
/// [`DefaultExecutor`]: struct.DefaultExecutor.html
pub fn spawn<F>(f: F) -> Spawn
where F: Future<Item = (), Error = ()> + 'static + Send
{
::tokio_executor::spawn(f);
Spawn(())
}
/// Like `spawn`, but compatible with futures 0.2
#[cfg(feature = "unstable-futures")]
pub fn spawn2<F>(f: F) -> Spawn
where F: futures2::Future<Item = (), Error = futures2::Never> + 'static + Send
{
::tokio_executor::spawn2(f);
Spawn(())
}
impl IntoFuture for Spawn {
type Future = FutureResult<(), ()>;
type Item = ();
type Error = ();
fn into_future(self) -> Self::Future {
future::ok(())
}
}
#[cfg(feature = "unstable-futures")]
impl futures2::IntoFuture for Spawn {
type Future = futures2::future::FutureResult<(), ()>;
type Item = ();
type Error = ();
fn into_future(self) -> Self::Future {
futures2::future::ok(())
}
}
-13
View File
@@ -1,13 +0,0 @@
//! Asynchronous filesystem manipulation operations.
//!
//! This module contains basic methods and types for manipulating the contents
//! of the local filesystem from within the context of the Tokio runtime.
//!
//! Unlike *most* other Tokio APIs, the filesystem APIs **must** be used from
//! the context of the Tokio runtime as they require Tokio specific features to
//! function.
pub use tokio_fs::{
file,
File,
};
-235
View File
@@ -1,235 +0,0 @@
//! A runtime for writing reliable, asynchronous, and slim applications.
//!
//! Tokio is an event-driven, non-blocking I/O platform for writing asynchronous
//! applications with the Rust programming language. At a high level, it
//! provides a few major components:
//!
//! * A multi threaded, work-stealing based task [scheduler][runtime].
//! * A [reactor][reactor] backed by the operating system's event queue (epoll, kqueue,
//! IOCP, etc...).
//! * Asynchronous [TCP and UDP][net] sockets.
//! * Asynchronous [filesystem][fs] operations.
//! * [Timer][timer] API for scheduling work in the future.
//!
//! Tokio is built using [futures] as the abstraction for managing the
//! complexity of asynchronous programming.
//!
//! Guide level documentation is found on the [website].
//!
//! [website]: https://tokio.rs/docs/getting-started/hello-world/
//! [futures]: http://docs.rs/futures
//!
//! # Examples
//!
//! A simple TCP echo server:
//!
//! ```no_run
//! extern crate tokio;
//!
//! use tokio::prelude::*;
//! use tokio::io::copy;
//! use tokio::net::TcpListener;
//!
//! fn main() {
//! // Bind the server's socket.
//! let addr = "127.0.0.1:12345".parse().unwrap();
//! let listener = TcpListener::bind(&addr)
//! .expect("unable to bind TCP listener");
//!
//! // Pull out a stream of sockets for incoming connections
//! let server = listener.incoming()
//! .map_err(|e| eprintln!("accept failed = {:?}", e))
//! .for_each(|sock| {
//! // Split up the reading and writing parts of the
//! // socket.
//! let (reader, writer) = sock.split();
//!
//! // A future that echos the data and returns how
//! // many bytes were copied...
//! let bytes_copied = copy(reader, writer);
//!
//! // ... after which we'll print what happened.
//! let handle_conn = bytes_copied.map(|amt| {
//! println!("wrote {:?} bytes", amt)
//! }).map_err(|err| {
//! eprintln!("IO error {:?}", err)
//! });
//!
//! // Spawn the future as a concurrent task.
//! tokio::spawn(handle_conn)
//! });
//!
//! // Start the Tokio runtime
//! tokio::run(server);
//! }
//! ```
#![doc(html_root_url = "https://docs.rs/tokio/0.1.5")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
#[macro_use]
extern crate futures;
extern crate mio;
extern crate tokio_io;
extern crate tokio_executor;
extern crate tokio_fs;
extern crate tokio_reactor;
extern crate tokio_threadpool;
extern crate tokio_timer;
extern crate tokio_tcp;
extern crate tokio_udp;
#[cfg(feature = "unstable-futures")]
extern crate futures2;
pub mod clock;
pub mod executor;
pub mod fs;
pub mod net;
pub mod reactor;
pub mod runtime;
pub mod timer;
pub mod util;
pub use executor::spawn;
#[cfg(feature = "unstable-futures")]
pub use executor::spawn2;
pub use runtime::run;
pub mod io {
//! Asynchronous I/O.
//!
//! This module is the asynchronous version of `std::io`. Primarily, it
//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which extend the
//! `Read` and `Write` traits of the standard library.
//!
//! # AsyncRead and AsyncWrite
//!
//! [`AsyncRead`] and [`AsyncWrite`] must only be implemented for
//! non-blocking I/O types that integrate with the futures type system. In
//! other words, these types must never block the thread, and instead the
//! current task is notified when the I/O resource is ready.
//!
//! # Standard input and output
//!
//! Tokio provides asynchronous APIs to standard [input], [output], and [error].
//! These APIs are very similar to the ones provided by `std`, but they also
//! implement [`AsyncRead`] and [`AsyncWrite`].
//!
//! Unlike *most* other Tokio APIs, the standard input / output APIs
//! **must** be used from the context of the Tokio runtime as they require
//! Tokio specific features to function.
//!
//! [input]: fn.stdin.html
//! [output]: fn.stdout.html
//! [error]: fn.stderr.html
//!
//! # Utility functions
//!
//! Utilities functions are provided for working with [`AsyncRead`] /
//! [`AsyncWrite`] types. For example, [`copy`] asynchronously copies all
//! data from a source to a destination.
//!
//! # `std` re-exports
//!
//! Additionally, [`Read`], [`Write`], [`Error`], [`ErrorKind`], and
//! [`Result`] are re-exported from `std::io` for ease of use.
//!
//! [`AsyncRead`]: trait.AsyncRead.html
//! [`AsyncWrite`]: trait.AsyncWrite.html
//! [`copy`]: fn.copy.html
//! [`Read`]: trait.Read.html
//! [`Write`]: trait.Write.html
//! [`Error`]: struct.Error.html
//! [`ErrorKind`]: enum.ErrorKind.html
//! [`Result`]: type.Result.html
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
// standard input, output, and error
pub use tokio_fs::{
stdin,
Stdin,
stdout,
Stdout,
stderr,
Stderr,
};
// Utils
pub use tokio_io::io::{
copy,
Copy,
flush,
Flush,
lines,
Lines,
read_exact,
ReadExact,
read_to_end,
ReadToEnd,
read_until,
ReadUntil,
ReadHalf,
shutdown,
Shutdown,
write_all,
WriteAll,
WriteHalf,
};
// Re-export io::Error so that users don't have to deal
// with conflicts when `use`ing `futures::io` and `std::io`.
pub use ::std::io::{
Error,
ErrorKind,
Result,
Read,
Write,
};
}
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
pub use util::{
FutureExt,
};
pub use ::std::io::{
Read,
Write,
};
pub use futures::{
Future,
future,
Stream,
stream,
Sink,
IntoFuture,
Async,
AsyncSink,
Poll,
task,
};
}
-41
View File
@@ -1,41 +0,0 @@
//! TCP/UDP bindings for `tokio`.
//!
//! This module contains the TCP/UDP networking types, similar to the standard
//! library, which can be used to implement networking protocols.
//!
//! # TCP
//!
//! Connecting to an address, via TCP, can be done using [`TcpStream`]'s
//! [`connect`] method, which returns [`ConnectFuture`]. `ConnectFuture`
//! implements a future which returns a `TcpStream`.
//!
//! To listen on an address [`TcpListener`] can be used. `TcpListener`'s
//! [`incoming`][incoming_method] method can be used to accept new connections.
//! It return the [`Incoming`] struct, which implements a stream which returns
//! `TcpStream`s.
//!
//! [`TcpStream`]: struct.TcpStream.html
//! [`connect`]: struct.TcpStream.html#method.connect
//! [`ConnectFuture`]: struct.ConnectFuture.html
//! [`TcpListener`]: struct.TcpListener.html
//! [incoming_method]: struct.TcpListener.html#method.incoming
//! [`Incoming`]: struct.Incoming.html
//!
//! # UDP
//!
//! The main struct for UDP is the [`UdpSocket`], which represents a UDP socket.
//! Reading and writing to it can be done using futures, which return the
//! [`RecvDgram`] and [`SendDgram`] structs respectively.
//!
//! For convenience it's also possible to convert raw datagrams into higher-level
//! frames.
//!
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`RecvDgram`]: struct.RecvDgram.html
//! [`SendDgram`]: struct.SendDgram.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`framed`]: struct.UdpSocket.html#method.framed
pub use tokio_tcp::{TcpStream, ConnectFuture};
pub use tokio_tcp::{TcpListener, Incoming};
pub use tokio_udp::{UdpSocket, UdpFramed, SendDgram, RecvDgram};
-149
View File
@@ -1,149 +0,0 @@
//! Event loop that drives Tokio I/O resources.
//!
//! This module contains [`Reactor`], which is the event loop that drives all
//! Tokio I/O resources. It is the reactor's job to receive events from the
//! operating system ([epoll], [kqueue], [IOCP], etc...) and forward them to
//! waiting tasks. It is the bridge between operating system and the futures
//! model.
//!
//! # Overview
//!
//! When using Tokio, all operations are asynchronous and represented by
//! futures. These futures, representing the application logic, are scheduled by
//! an executor (see [runtime model] for more details). Executors wait for
//! notifications before scheduling the future for execution time, i.e., nothing
//! happens until an event is received indicating that the task can make
//! progress.
//!
//! The reactor receives events from the operating system and notifies the
//! executor.
//!
//! Let's start with a basic example, establishing a TCP connection.
//!
//! ```rust
//! # extern crate tokio;
//! # fn dox() {
//! use tokio::prelude::*;
//! use tokio::net::TcpStream;
//!
//! let addr = "93.184.216.34:9243".parse().unwrap();
//!
//! let connect_future = TcpStream::connect(&addr);
//!
//! let task = connect_future
//! .and_then(|socket| {
//! println!("successfully connected");
//! Ok(())
//! })
//! .map_err(|e| println!("failed to connect; err={:?}", e));
//!
//! tokio::run(task);
//! # }
//! # fn main() {}
//! ```
//!
//! Establishing a TCP connection usually cannot be completed immediately.
//! [`TcpStream::connect`] does not block the current thread. Instead, it
//! returns a [future][connect-future] that resolves once the TCP connection has
//! been established. The connect future itself has no way of knowing when the
//! TCP connection has been established.
//!
//! Before returning the future, [`TcpStream::connect`] registers the socket
//! with a reactor. This registration process, handled by [`Registration`], is
//! what links the [`TcpStream`] with the [`Reactor`] instance. At this point,
//! the reactor starts listening for connection events from the operating system
//! for that socket.
//!
//! Once the connect future is passed to [`tokio::run`], it is spawned onto a
//! thread pool. The thread pool waits until it is notified that the connection
//! has completed.
//!
//! When the TCP connection is established, the reactor receives an event from
//! the operating system. It then notifies the thread pool, telling it that the
//! connect future can complete. At this point, the thread pool will schedule
//! the task to run on one of its worker threads. This results in the `and_then`
//! closure to get executed.
//!
//! ## Lazy registration
//!
//! Notice how the snippet above does not explicitly reference a reactor. When
//! [`TcpStream::connect`] is called, it registers the socket with a reactor,
//! but no reactor is specified. This works because the registration process
//! mentioned above is actually lazy. It doesn't *actually* happen in the
//! [`connect`] function. Instead, the registration is established the first
//! time that the task is polled (again, see [runtime model]).
//!
//! A reactor instance is automatically made available when using the Tokio
//! [runtime], which is done using [`tokio::run`]. The Tokio runtime's executor
//! sets a thread-local variable referencing the associated [`Reactor`] instance
//! and [`Handle::current`] (used by [`Registration`]) returns the reference.
//!
//! ## Implementation
//!
//! The reactor implementation uses [`mio`] to interface with the operating
//! system's event queue. A call to [`Reactor::poll`] results in in a single
//! call to [`Poll::poll`] which in turn results in a single call to the
//! operating system's selector.
//!
//! The reactor maintains state for each registered I/O resource. This tracks
//! the executor task to notify when events are provided by the operating
//! system's selector. This state is stored in a `Sync` data structure and
//! referenced by [`Registration`]. When the [`Registration`] instance is
//! dropped, this state is cleaned up. Because the state is stored in a `Sync`
//! data structure, the [`Registration`] instance is able to be moved to other
//! threads.
//!
//! By default, a runtime's default reactor runs on a background thread. This
//! ensures that application code cannot significantly impact the reactor's
//! responsiveness.
//!
//! ## Integrating with the reactor
//!
//! Tokio comes with a number of I/O resources, like TCP and UDP sockets, that
//! automatically integrate with the reactor. However, library authors or
//! applications may wish to implement their own resources that are also backed
//! by the reactor.
//!
//! There are a couple of ways to do this.
//!
//! If the custom I/O resource implements [`mio::Evented`] and implements
//! [`std::Read`] and / or [`std::Write`], then [`PollEvented`] is the most
//! suited.
//!
//! Otherwise, [`Registration`] can be used directly. This provides the lowest
//! level primitive needed for integrating with the reactor: a stream of
//! readiness events.
//!
//! [`Reactor`]: struct.Reactor.html
//! [`Registration`]: struct.Registration.html
//! [runtime model]: https://tokio.rs/docs/getting-started/runtime-model/
//! [epoll]: http://man7.org/linux/man-pages/man7/epoll.7.html
//! [kqueue]: https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2
//! [IOCP]: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365198(v=vs.85).aspx
//! [`TcpStream::connect`]: ../net/struct.TcpStream.html#method.connect
//! [`connect`]: ../net/struct.TcpStream.html#method.connect
//! [connect-future]: ../net/struct.ConnectFuture.html
//! [`tokio::run`]: ../runtime/fn.run.html
//! [`TcpStream`]: ../net/struct.TcpStream.html
//! [runtime]: ../runtime
//! [`Handle::current`]: struct.Handle.html#method.current
//! [`mio`]: https://github.com/carllerche/mio
//! [`Reactor::poll`]: struct.Reactor.html#method.poll
//! [`Poll::poll`]: https://docs.rs/mio/0.6/mio/struct.Poll.html#method.poll
//! [`mio::Evented`]: https://docs.rs/mio/0.6/mio/trait.Evented.html
//! [`PollEvented`]: struct.PollEvented.html
//! [`std::Read`]: https://doc.rust-lang.org/std/io/trait.Read.html
//! [`std::Write`]: https://doc.rust-lang.org/std/io/trait.Write.html
pub use tokio_reactor::{
Reactor,
Handle,
Background,
Turn,
Registration,
PollEvented as PollEvented2,
};
mod poll_evented;
#[allow(deprecated)]
pub use self::poll_evented::PollEvented;
-539
View File
@@ -1,539 +0,0 @@
//! Readiness tracking streams, backing I/O objects.
//!
//! This module contains the core type which is used to back all I/O on object
//! in `tokio-core`. The `PollEvented` type is the implementation detail of
//! all I/O. Each `PollEvented` manages registration with a reactor,
//! acquisition of a token, and tracking of the readiness state on the
//! underlying I/O primitive.
#![allow(deprecated, warnings)]
use std::fmt;
use std::io::{self, Read, Write};
use std::sync::Mutex;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::{task, Async, Poll};
use mio::event::Evented;
use mio::Ready;
use tokio_io::{AsyncRead, AsyncWrite};
use reactor::{Handle, Registration};
#[deprecated(since = "0.1.2", note = "PollEvented2 instead")]
#[doc(hidden)]
pub struct PollEvented<E> {
io: E,
inner: Inner,
handle: Handle,
}
struct Inner {
registration: Mutex<Registration>,
/// Currently visible read readiness
read_readiness: AtomicUsize,
/// Currently visible write readiness
write_readiness: AtomicUsize,
}
impl<E: fmt::Debug> fmt::Debug for PollEvented<E> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("PollEvented")
.field("io", &self.io)
.finish()
}
}
impl<E> PollEvented<E> {
/// Creates a new readiness stream associated with the provided
/// `loop_handle` and for the given `source`.
pub fn new(io: E, handle: &Handle) -> io::Result<PollEvented<E>>
where E: Evented,
{
let registration = Registration::new();
registration.register(&io)?;
Ok(PollEvented {
io: io,
inner: Inner {
registration: Mutex::new(registration),
read_readiness: AtomicUsize::new(0),
write_readiness: AtomicUsize::new(0),
},
handle: handle.clone(),
})
}
/// Tests to see if this source is ready to be read from or not.
///
/// If this stream is not ready for a read then `Async::NotReady` will be
/// returned and the current task will be scheduled to receive a
/// notification when the stream is readable again. In other words, this
/// method is only safe to call from within the context of a future's task,
/// typically done in a `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::readable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_read(&mut self) -> Async<()> {
if self.poll_read2().is_ready() {
return ().into();
}
Async::NotReady
}
fn poll_read2(&self) -> Async<Ready> {
let r = self.inner.registration.lock().unwrap();
// Load the cached readiness
match self.inner.read_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_read_ready().unwrap() {
n |= ready2usize(ready);
self.inner.read_readiness.store(n, Relaxed);
}
return usize2ready(n).into();
}
}
let ready = match r.poll_read_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.read_readiness.store(ready2usize(ready), Relaxed);
ready.into()
}
/// Tests to see if this source is ready to be written to or not.
///
/// If this stream is not ready for a write then `Async::NotReady` will be returned
/// and the current task will be scheduled to receive a notification when
/// the stream is writable again. In other words, this method is only safe
/// to call from within the context of a future's task, typically done in a
/// `Future::poll` method.
///
/// This is mostly equivalent to `self.poll_ready(Ready::writable())`.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_write(&mut self) -> Async<()> {
let r = self.inner.registration.lock().unwrap();
match self.inner.write_readiness.load(Relaxed) {
0 => {}
mut n => {
// Check what's new with the reactor.
if let Some(ready) = r.take_write_ready().unwrap() {
n |= ready2usize(ready);
self.inner.write_readiness.store(n, Relaxed);
}
return ().into();
}
}
let ready = match r.poll_write_ready().unwrap() {
Async::Ready(r) => r,
_ => return Async::NotReady,
};
// Cache the value
self.inner.write_readiness.store(ready2usize(ready), Relaxed);
().into()
}
/// Test to see whether this source fulfills any condition listed in `mask`
/// provided.
///
/// The `mask` given here is a mio `Ready` set of possible events. This can
/// contain any events like read/write but also platform-specific events
/// such as hup and error. The `mask` indicates events that are interested
/// in being ready.
///
/// If any event in `mask` is ready then it is returned through
/// `Async::Ready`. The `Ready` set returned is guaranteed to not be empty
/// and contains all events that are currently ready in the `mask` provided.
///
/// If no events are ready in the `mask` provided then the current task is
/// scheduled to receive a notification when any of them become ready. If
/// the `writable` event is contained within `mask` then this
/// `PollEvented`'s `write` task will be blocked and otherwise the `read`
/// task will be blocked. This is generally only relevant if you're working
/// with this `PollEvented` object on multiple tasks.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn poll_ready(&mut self, mask: Ready) -> Async<Ready> {
let mut ret = Ready::empty();
if mask.is_empty() {
return ret.into();
}
if mask.is_writable() {
if self.poll_write().is_ready() {
ret = Ready::writable();
}
}
let mask = mask - Ready::writable();
if !mask.is_empty() {
if let Async::Ready(v) = self.poll_read2() {
ret |= v & mask;
}
}
if ret.is_empty() {
if mask.is_writable() {
let _ = self.need_write();
}
if mask.is_readable() {
let _ = self.need_read();
}
Async::NotReady
} else {
ret.into()
}
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer readable, but it needs to be.
///
/// This function, like `poll_read`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// readable, typically because a "would block" error was seen.
///
/// *All* readiness bits associated with this stream except the writable bit
/// will be reset when this method is called. The current task is then
/// scheduled to receive a notification whenever anything changes other than
/// the writable bit. Note that this typically just means the readable bit
/// is used here, but if you're using a custom I/O object for events like
/// hup/error this may also be relevant.
///
/// Note that it is also only valid to call this method if `poll_read`
/// previously indicated that the object is readable. That is, this function
/// must always be paired with calls to `poll_read` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_read(&mut self) -> io::Result<()> {
self.inner.read_readiness.store(0, Relaxed);
if self.poll_read().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Indicates to this source of events that the corresponding I/O object is
/// no longer writable, but it needs to be.
///
/// This function, like `poll_write`, is only safe to call from the context
/// of a future's task (typically in a `Future::poll` implementation). It
/// informs this readiness stream that the underlying object is no longer
/// writable, typically because a "would block" error was seen.
///
/// The flag indicating that this stream is writable is unset and the
/// current task is scheduled to receive a notification when the stream is
/// then again writable.
///
/// Note that it is also only valid to call this method if `poll_write`
/// previously indicated that the object is writable. That is, this function
/// must always be paired with calls to `poll_write` previously.
///
/// # Errors
///
/// This function will return an error if the `Reactor` that this `PollEvented`
/// is associated with has gone away (been destroyed). The error means that
/// the ambient futures task could not be scheduled to receive a
/// notification and typically means that the error should be propagated
/// outwards.
///
/// # Panics
///
/// This function will panic if called outside the context of a future's
/// task.
pub fn need_write(&mut self) -> io::Result<()> {
self.inner.write_readiness.store(0, Relaxed);
if self.poll_write().is_ready() {
// Notify the current task
task::current().notify();
}
Ok(())
}
/// Returns a reference to the event loop handle that this readiness stream
/// is associated with.
pub fn handle(&self) -> &Handle {
&self.handle
}
/// Returns a shared reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_ref(&self) -> &E {
&self.io
}
/// Returns a mutable reference to the underlying I/O object this readiness
/// stream is wrapping.
pub fn get_mut(&mut self) -> &mut E {
&mut self.io
}
/// Consumes the `PollEvented` and returns the underlying I/O object
pub fn into_inner(self) -> E {
self.io
}
/// Deregisters this source of events from the reactor core specified.
///
/// This method can optionally be called to unregister the underlying I/O
/// object with the event loop that the `handle` provided points to.
/// Typically this method is not required as this automatically happens when
/// `E` is dropped, but for some use cases the `E` object doesn't represent
/// an owned reference, so dropping it won't automatically unregister with
/// the event loop.
///
/// This consumes `self` as it will no longer provide events after the
/// method is called, and will likely return an error if this `PollEvented`
/// was created on a separate event loop from the `handle` specified.
pub fn deregister(&self) -> io::Result<()>
where E: Evented,
{
self.inner.registration.lock().unwrap()
.deregister(&self.io)
}
}
impl<E: Read> Read for PollEvented<E> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_read() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().read(buf);
if is_wouldblock(&r) {
self.need_read()?;
}
return r
}
}
impl<E: Write> Write for PollEvented<E> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().write(buf);
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
fn flush(&mut self) -> io::Result<()> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
}
let r = self.get_mut().flush();
if is_wouldblock(&r) {
self.need_write()?;
}
return r
}
}
impl<E: Read> AsyncRead for PollEvented<E> {
}
impl<E: Write> AsyncWrite for PollEvented<E> {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
fn is_wouldblock<T>(r: &io::Result<T>) -> bool {
match *r {
Ok(_) => false,
Err(ref e) => e.kind() == io::ErrorKind::WouldBlock,
}
}
const READ: usize = 1 << 0;
const WRITE: usize = 1 << 1;
fn ready2usize(ready: Ready) -> usize {
let mut bits = 0;
if ready.is_readable() {
bits |= READ;
}
if ready.is_writable() {
bits |= WRITE;
}
bits | platform::ready2usize(ready)
}
fn usize2ready(bits: usize) -> Ready {
let mut ready = Ready::empty();
if bits & READ != 0 {
ready.insert(Ready::readable());
}
if bits & WRITE != 0 {
ready.insert(Ready::writable());
}
ready | platform::usize2ready(bits)
}
#[cfg(unix)]
mod platform {
use mio::Ready;
use mio::unix::UnixReady;
const HUP: usize = 1 << 2;
const ERROR: usize = 1 << 3;
const AIO: usize = 1 << 4;
const LIO: usize = 1 << 5;
#[cfg(any(target_os = "dragonfly", target_os = "freebsd"))]
fn is_aio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_aio()
}
#[cfg(not(any(target_os = "dragonfly", target_os = "freebsd")))]
fn is_aio(_ready: &Ready) -> bool {
false
}
#[cfg(target_os = "freebsd")]
fn is_lio(ready: &Ready) -> bool {
UnixReady::from(*ready).is_lio()
}
#[cfg(not(target_os = "freebsd"))]
fn is_lio(_ready: &Ready) -> bool {
false
}
pub fn ready2usize(ready: Ready) -> usize {
let ready = UnixReady::from(ready);
let mut bits = 0;
if is_aio(&ready) {
bits |= AIO;
}
if is_lio(&ready) {
bits |= LIO;
}
if ready.is_error() {
bits |= ERROR;
}
if ready.is_hup() {
bits |= HUP;
}
bits
}
#[cfg(any(target_os = "dragonfly", target_os = "freebsd", target_os = "ios",
target_os = "macos"))]
fn usize2ready_aio(ready: &mut UnixReady) {
ready.insert(UnixReady::aio());
}
#[cfg(not(any(target_os = "dragonfly",
target_os = "freebsd", target_os = "ios", target_os = "macos")))]
fn usize2ready_aio(_ready: &mut UnixReady) {
// aio not available here → empty
}
#[cfg(target_os = "freebsd")]
fn usize2ready_lio(ready: &mut UnixReady) {
ready.insert(UnixReady::lio());
}
#[cfg(not(target_os = "freebsd"))]
fn usize2ready_lio(_ready: &mut UnixReady) {
// lio not available here → empty
}
pub fn usize2ready(bits: usize) -> Ready {
let mut ready = UnixReady::from(Ready::empty());
if bits & AIO != 0 {
usize2ready_aio(&mut ready);
}
if bits & LIO != 0 {
usize2ready_lio(&mut ready);
}
if bits & HUP != 0 {
ready.insert(UnixReady::hup());
}
if bits & ERROR != 0 {
ready.insert(UnixReady::error());
}
ready.into()
}
}
#[cfg(windows)]
mod platform {
use mio::Ready;
pub fn all() -> Ready {
// No platform-specific Readinesses for Windows
Ready::empty()
}
pub fn hup() -> Ready {
Ready::empty()
}
pub fn ready2usize(_r: Ready) -> usize {
0
}
pub fn usize2ready(_r: usize) -> Ready {
Ready::empty()
}
}
-148
View File
@@ -1,148 +0,0 @@
use runtime::{Inner, Runtime};
use reactor::Reactor;
use std::io;
use tokio_reactor;
use tokio_threadpool::Builder as ThreadPoolBuilder;
use tokio_threadpool::park::DefaultPark;
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
/// Builds Tokio Runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate tokio_threadpool;
/// # use tokio::runtime::Builder;
///
/// # pub fn main() {
/// // create and configure ThreadPool
/// let mut threadpool_builder = tokio_threadpool::Builder::new();
/// threadpool_builder
/// .name_prefix("my-runtime-worker-")
/// .pool_size(4);
///
/// // build Runtime
/// let runtime = Builder::new()
/// .threadpool_builder(threadpool_builder)
/// .build();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
#[derive(Debug)]
pub struct Builder {
/// Thread pool specific builder
threadpool_builder: ThreadPoolBuilder,
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
let mut threadpool_builder = ThreadPoolBuilder::new();
threadpool_builder.name_prefix("tokio-runtime-worker-");
Builder {
threadpool_builder,
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Set builder to set up the thread pool instance.
pub fn threadpool_builder(&mut self, val: ThreadPoolBuilder) -> &mut Self {
self.threadpool_builder = val;
self
}
/// Create the configured `Runtime`.
///
/// The returned `ThreadPool` instance is ready to spawn tasks.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::runtime::Builder;
/// # pub fn main() {
/// let runtime = Builder::new().build().unwrap();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
pub fn build(&mut self) -> io::Result<Runtime> {
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
// Get a handle to the clock for the runtime.
let clock1 = self.clock.clone();
let clock2 = clock1.clone();
let timers = Arc::new(Mutex::new(HashMap::<_, timer::Handle>::new()));
let t1 = timers.clone();
// Spawn a reactor on a background thread.
let reactor = Reactor::new()?.background()?;
// Get a handle to the reactor.
let reactor_handle = reactor.handle().clone();
let pool = self.threadpool_builder
.around_worker(move |w, enter| {
let timer_handle = t1.lock().unwrap()
.get(w.id()).unwrap()
.clone();
tokio_reactor::with_default(&reactor_handle, enter, |enter| {
clock::with_default(&clock1, enter, |enter| {
timer::with_default(&timer_handle, enter, |_| {
w.run();
});
})
});
})
.custom_park(move |worker_id| {
// Create a new timer
let timer = Timer::new_with_now(DefaultPark::new(), clock2.clone());
timers.lock().unwrap()
.insert(worker_id.clone(), timer.handle());
timer
})
.build();
Ok(Runtime {
inner: Some(Inner {
reactor,
pool,
}),
})
}
}
-88
View File
@@ -1,88 +0,0 @@
use executor::current_thread::CurrentThread;
use runtime::current_thread::Runtime;
use tokio_reactor::Reactor;
use tokio_timer::clock::Clock;
use tokio_timer::timer::Timer;
use std::io;
/// Builds a Single-threaded runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// extern crate tokio;
/// extern crate tokio_timer;
///
/// use tokio::runtime::current_thread::Builder;
/// use tokio_timer::clock::Clock;
///
/// # pub fn main() {
/// // build Runtime
/// let runtime = Builder::new()
/// .clock(Clock::new())
/// .build();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
#[derive(Debug)]
pub struct Builder {
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
Builder {
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Create the configured `Runtime`.
pub fn build(&mut self) -> io::Result<Runtime> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new_with_now(reactor, self.clock.clone());
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor to generate some new stimuli for the
// futures to continue in their life.
let executor = CurrentThread::new_with_park(timer);
let runtime = Runtime::new2(
reactor_handle,
timer_handle,
self.clock.clone(),
executor);
Ok(runtime)
}
}
-70
View File
@@ -1,70 +0,0 @@
//! A runtime implementation that runs everything on the current thread.
//!
//! [`current_thread::Runtime`][rt] is similar to the primary
//! [`Runtime`][concurrent-rt] except that it runs all components on the current
//! thread instead of using a thread pool. This means that it is able to spawn
//! futures that do not implement `Send`.
//!
//! Same as the default [`Runtime`][concurrent-rt], the
//! [`current_thread::Runtime`][rt] includes:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! Note that [`current_thread::Runtime`][rt] does not implement `Send` itself
//! and cannot be safely moved to other threads.
//!
//! # Spawning from other threads
//!
//! While [`current_thread::Runtime`][rt] does not implement `Send` and cannot
//! safely be moved to other threads, it provides a `Handle` that can be sent
//! to other threads and allows to spawn new tasks from there.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//! use std::thread;
//!
//! # fn main() {
//! let mut runtime = Runtime::new().unwrap();
//! let handle = runtime.handle();
//!
//! thread::spawn(move || {
//! handle.spawn(future::ok(()));
//! }).join().unwrap();
//!
//! # /*
//! runtime.run().unwrap();
//! # */
//! # }
//! ```
//!
//! # Examples
//!
//! Creating a new `Runtime` and running a future `f` until its completion and
//! returning its result.
//!
//! ```
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//!
//! let mut runtime = Runtime::new().unwrap();
//!
//! // Use the runtime...
//! // runtime.block_on(f); // where f is a future
//! ```
//!
//! [rt]: struct.Runtime.html
//! [concurrent-rt]: ../struct.Runtime.html
//! [chan]: https://docs.rs/futures/0.1/futures/sync/mpsc/fn.channel.html
mod builder;
mod runtime;
pub use self::builder::Builder;
pub use self::runtime::{Runtime, Handle};
-185
View File
@@ -1,185 +0,0 @@
use executor::current_thread::{self, CurrentThread};
use executor::current_thread::Handle as ExecutorHandle;
use runtime::current_thread::Builder;
use tokio_reactor::{self, Reactor};
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
use tokio_executor;
use futures::Future;
use std::io;
/// Single-threaded runtime provides a way to start reactor
/// and executor on the current thread.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
#[derive(Debug)]
pub struct Runtime {
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>,
}
/// Handle to spawn a future on the corresponding `CurrentThread` runtime instance
#[derive(Debug, Clone)]
pub struct Handle(ExecutorHandle);
impl Handle {
/// Spawn a future onto the `CurrentThread` runtime instance corresponding to this handle
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the `CurrentThread`
/// instance of the `Handle` does not exist anymore.
pub fn spawn<F>(&self, future: F) -> Result<(), tokio_executor::SpawnError>
where F: Future<Item = (), Error = ()> + Send + 'static {
self.0.spawn(future)
}
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
inner: current_thread::RunError,
}
impl Runtime {
/// Returns a new runtime initialized with default configuration values.
pub fn new() -> io::Result<Runtime> {
Builder::new().build()
}
pub(super) fn new2(
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>) -> Runtime
{
Runtime {
reactor_handle,
timer_handle,
clock,
executor,
}
}
/// Get a new handle to spawn futures on the single-threaded Tokio runtime
///
/// Different to the runtime itself, the handle can be sent to different
/// threads.
pub fn handle(&self) -> Handle {
Handle(self.executor.handle().clone())
}
/// Spawn a future onto the single-threaded Tokio runtime.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::current_thread::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("running on the runtime");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.spawn(future);
self
}
/// Runs the provided future, blocking the current thread until the future
/// completes.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed. Once the function returns, any uncompleted futures
/// remain pending in the `Runtime` instance. These futures will not run
/// until `block_on` or `run` is called again.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution by calling `block_on` or `run`.
pub fn block_on<F>(&mut self, f: F) -> Result<F::Item, F::Error>
where F: Future
{
self.enter(|executor| {
// Run the provided future
let ret = executor.block_on(f);
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
})
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.enter(|executor| executor.run())
.map_err(|e| RunError {
inner: e,
})
}
fn enter<F, R>(&mut self, f: F) -> R
where F: FnOnce(&mut current_thread::Entered<Timer<Reactor>>) -> R
{
let Runtime {
ref reactor_handle,
ref timer_handle,
ref clock,
ref mut executor,
..
} = *self;
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure
// and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
clock::with_default(clock, enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the
// current single-threaded executor when used. This is a trick,
// because we need two mutable references to the executor (one
// to run the provided future, another to install as the default
// one). We use the fake one here as the default one.
let mut default_executor = current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
f(&mut executor)
})
})
})
})
}
}
-495
View File
@@ -1,495 +0,0 @@
//! A batteries included runtime for applications using Tokio.
//!
//! Applications using Tokio require some runtime support in order to work:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
//!
//! [`Runtime`] bundles all of these various runtime components into a single
//! handle that can be started and shutdown together, eliminating the necessary
//! boilerplate to run a Tokio application.
//!
//! Most applications wont need to use [`Runtime`] directly. Instead, they will
//! use the [`run`] function, which uses [`Runtime`] under the hood.
//!
//! Creating a [`Runtime`] does the following:
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//! * Run an instance of [`Timer`] **per** thread pool worker thread.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! A timer per thread pool worker thread is used to minimize the amount of
//! synchronization that is required for working with the timer.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
//! "seed" the application, blocking the thread until the runtime becomes
//! [idle].
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! tokio::run(server);
//! # }
//! # pub fn main() {}
//! ```
//!
//! In this function, the `run` function blocks until the runtime becomes idle.
//! See [`shutdown_on_idle`][idle] for more shutdown details.
//!
//! From within the context of the runtime, additional tasks are spawned using
//! the [`tokio::spawn`] function. Futures spawned using this function will be
//! executed on the same thread pool used by the [`Runtime`].
//!
//! A [`Runtime`] instance can also be used directly.
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::runtime::Runtime;
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! // Create the runtime
//! let mut rt = Runtime::new().unwrap();
//!
//! // Spawn the server task
//! rt.spawn(server);
//!
//! // Wait until the runtime becomes idle and shut it down.
//! rt.shutdown_on_idle()
//! .wait().unwrap();
//! # }
//! # pub fn main() {}
//! ```
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../timer/index.html
//! [`Runtime`]: struct.Runtime.html
//! [`Reactor`]: ../reactor/struct.Reactor.html
//! [`ThreadPool`]: ../executor/thread_pool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
//! [`Timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/timer/struct.Timer.html
mod builder;
pub mod current_thread;
mod shutdown;
mod task_executor;
pub use self::builder::Builder;
pub use self::shutdown::Shutdown;
pub use self::task_executor::TaskExecutor;
use reactor::{Background, Handle};
use std::io;
use tokio_threadpool as threadpool;
use futures;
use futures::future::Future;
#[cfg(feature = "unstable-futures")]
use futures2;
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// Instances of `Runtime` can be created using [`new`] or [`Builder`]. However,
/// most users will use [`tokio::run`], which uses a `Runtime` internally.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
/// [`new`]: #method.new
/// [`Builder`]: struct.Builder.html
/// [`tokio::run`]: fn.run.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
#[derive(Debug)]
struct Inner {
/// Reactor running on a background thread.
reactor: Background,
/// Task execution pool.
pool: threadpool::ThreadPool,
}
// ===== impl Runtime =====
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// This function is used to bootstrap the execution of a Tokio application. It
/// does the following:
///
/// * Start the Tokio runtime using a default configuration.
/// * Spawn the given future onto the thread pool.
/// * Block the current thread until the runtime shuts down.
///
/// Note that the function will not return immediately once `future` has
/// completed. Instead it waits for the entire runtime to become idle.
///
/// See the [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if called from the context of an executor.
///
/// [mod]: ../index.html
pub fn run<F>(future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
let mut runtime = Runtime::new().unwrap();
runtime.spawn(future);
runtime.shutdown_on_idle().wait().unwrap();
}
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// Identical to `run` but works with futures 0.2-style futures.
#[cfg(feature = "unstable-futures")]
pub fn run2<F>(future: F)
where F: futures2::Future<Item = (), Error = futures2::Never> + Send + 'static,
{
let mut runtime = Runtime::new().unwrap();
runtime.spawn2(future);
runtime.shutdown_on_idle().wait().unwrap();
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// This results in a reactor, thread pool, and timer being initialized. The
/// thread pool will not spawn any worker threads until it needs to, i.e.
/// tasks are scheduled to run.
///
/// Most users will not need to call this function directly, instead they
/// will use [`tokio::run`](fn.run.html).
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// Creating a new `Runtime` with default configuration values.
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
Builder::new().build()
}
#[deprecated(since = "0.1.5", note = "use `reactor` instead")]
#[doc(hidden)]
pub fn handle(&self) -> &Handle {
self.reactor()
}
/// Return a reference to the reactor handle for this runtime instance.
///
/// The returned handle reference can be cloned in order to get an owned
/// value of the handle. This handle can be used to initialize I/O resources
/// (like TCP or UDP sockets) that will not be used on the runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let reactor_handle = rt.reactor().clone();
///
/// // use `reactor_handle`
/// ```
pub fn reactor(&self) -> &Handle {
self.inner().reactor.handle()
}
/// Return a handle to the runtime's executor.
///
/// The returned handle can be used to spawn tasks that run on this runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let executor_handle = rt.executor();
///
/// // use `executor_handle`
/// ```
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
}
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner_mut().pool.sender().spawn(future).unwrap();
self
}
/// Spawn a futures 0.2-style future onto the Tokio runtime.
///
/// Otherwise identical to `spawn`
#[cfg(feature = "unstable-futures")]
pub fn spawn2<F>(&mut self, future: F) -> &mut Self
where F: futures2::Future<Item = (), Error = futures2::Never> + Send + 'static,
{
futures2::executor::Executor::spawn(
self.inner_mut().pool.sender_mut(), Box::new(future)
).unwrap();
self
}
/// Run a future to completion on the Tokio runtime.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, and yielding its resolved result. Any tasks or timers which
/// the future spawns internally will be executed on the runtime.
///
/// This method should not be called from an asynchrounous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on<F, R, E>(&mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
rx.wait().unwrap()
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function can be used to perform a graceful shutdown of the runtime.
///
/// The runtime enters an idle state once **all** of the following occur.
///
/// * The thread pool has no tasks to execute, i.e., all tasks that were
/// spawned have completed.
/// * The reactor is not managing any I/O resources.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_on_idle()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_on_idle().and_then(|_| {
reactor.shutdown_on_idle()
})
});
Shutdown { inner }
}
/// Signals the runtime to shutdown immediately.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function will forcibly shutdown the runtime, causing any
/// in-progress work to become canceled. The shutdown steps are:
///
/// * Drain any scheduled work queues.
/// * Drop any futures that have not yet completed.
/// * Drop the reactor.
///
/// Once the reactor has dropped, any outstanding I/O resources bound to
/// that reactor will no longer function. Calling any method on them will
/// result in an error.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
Shutdown::shutdown_now(inner)
}
fn inner(&self) -> &Inner {
self.inner.as_ref().unwrap()
}
fn inner_mut(&mut self) -> &mut Inner {
self.inner.as_mut().unwrap()
}
}
impl Drop for Runtime {
fn drop(&mut self) {
if let Some(inner) = self.inner.take() {
let shutdown = Shutdown::shutdown_now(inner);
let _ = shutdown.wait();
}
}
}
-46
View File
@@ -1,46 +0,0 @@
use runtime::Inner;
use std::fmt;
use futures::{Future, Poll};
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
pub(super) inner: Box<Future<Item = (), Error = ()> + Send>,
}
impl Shutdown {
pub(super) fn shutdown_now(inner: Inner) -> Self {
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_now().and_then(|_| {
reactor.shutdown_now()
.then(|_| {
Ok(())
})
})
});
Shutdown { inner }
}
}
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
-98
View File
@@ -1,98 +0,0 @@
use tokio_threadpool::Sender;
use futures::future::{self, Future};
#[cfg(feature = "unstable-futures")]
use futures2;
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
pub(super) inner: Sender,
}
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
#[cfg(feature = "unstable-futures")]
fn spawn2(&mut self, future: Box<futures2::Future<Item = (), Error = futures2::Never> + Send>)
-> Result<(), futures2::executor::SpawnError>
{
self.inner.spawn2(future)
}
}
#[cfg(feature = "unstable-futures")]
type Task2 = Box<futures2::Future<Item = (), Error = futures2::Never> + Send>;
#[cfg(feature = "unstable-futures")]
impl futures2::executor::Executor for TaskExecutor {
fn spawn(&mut self, f: Task2) -> Result<(), futures2::executor::SpawnError> {
futures2::executor::Executor::spawn(&mut self.inner, f)
}
fn status(&self) -> Result<(), futures2::executor::SpawnError> {
futures2::executor::Executor::status(&self.inner)
}
}
-86
View File
@@ -1,86 +0,0 @@
//! Utilities for tracking time.
//!
//! This module provides a number of types for executing code after a set period
//! of time.
//!
//! * [`Delay`][Delay] is a future that does no work and completes at a specific `Instant`
//! in time.
//!
//! * [`Interval`][Interval] is a stream yielding a value at a fixed period. It
//! is initialized with a `Duration` and repeatedly yields each time the
//! duration elapses.
//!
//! * [`Deadline`][Deadline] wraps a future, requiring that it completes before
//! a specified `Instant` in time. If the future does not complete in time,
//! then it is canceled and an error is returned.
//!
//! These types are sufficient for handling a large number of scenarios
//! involving time.
//!
//! These types must be used from within the context of the
//! [`Runtime`][runtime] or a timer context must be setup explicitly. See the
//! [`tokio-timer`][tokio-timer] crate for more details on how to setup a timer
//! context.
//!
//! # Examples
//!
//! Wait 100ms and print "Hello World!"
//!
//! ```
//! use tokio::prelude::*;
//! use tokio::timer::Delay;
//!
//! use std::time::{Duration, Instant};
//!
//! let when = Instant::now() + Duration::from_millis(100);
//!
//! tokio::run({
//! Delay::new(when)
//! .map_err(|e| panic!("timer failed; err={:?}", e))
//! .and_then(|_| {
//! println!("Hello world!");
//! Ok(())
//! })
//! })
//! ```
//!
//! Require that an operation takes no more than 300ms. Note that this uses the
//! [`deadline`][ext] function on the [`FutureExt`][ext] trait. This trait is
//! included in the prelude.
//!
//! ```
//! # extern crate futures;
//! # extern crate tokio;
//! use tokio::prelude::*;
//!
//! use std::time::{Duration, Instant};
//!
//! fn long_op() -> Box<Future<Item = (), Error = ()> + Send> {
//! // ...
//! # Box::new(futures::future::ok(()))
//! }
//!
//! # fn main() {
//! let when = Instant::now() + Duration::from_millis(300);
//!
//! tokio::run({
//! long_op()
//! .deadline(when)
//! .map_err(|e| {
//! println!("operation timed out");
//! })
//! })
//! # }
//! ```
//!
//! [runtime]: ../runtime/struct.Runtime.html
//! [tokio-timer]: https://docs.rs/tokio-timer
//! [ext]: ../util/trait.FutureExt.html#method.deadline
pub use tokio_timer::{
Deadline,
DeadlineError,
Error,
Interval,
Delay,
};
-61
View File
@@ -1,61 +0,0 @@
use tokio_timer::Deadline;
use futures::Future;
use std::time::Instant;
/// An extension trait for `Future` that provides a variety of convenient
/// combinator functions.
///
/// Currently, there only is a [`deadline`] function, but this will increase
/// over time.
///
/// Users are not expected to implement this trait. All types that implement
/// `Future` already implement `FutureExt`.
///
/// This trait can be imported directly or via the Tokio prelude: `use
/// tokio::prelude::*`.
///
/// [`deadline`]: #method.deadline
pub trait FutureExt: Future {
/// Creates a new future which allows `self` until `deadline`.
///
/// This combinator creates a new future which wraps the receiving future
/// with a deadline. The returned future is allowed to execute until it
/// completes or `deadline` is reached, whichever happens first.
///
/// If the future completes before `deadline` then the future will resolve
/// with that item. Otherwise the future will resolve to an error once
/// `deadline` is reached.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// use tokio::prelude::*;
/// use std::time::{Duration, Instant};
/// # use futures::future::{self, FutureResult};
///
/// # fn long_future() -> FutureResult<(), ()> {
/// # future::ok(())
/// # }
/// #
/// # fn main() {
/// let future = long_future()
/// .deadline(Instant::now() + Duration::from_secs(1))
/// .map_err(|e| println!("error = {:?}", e));
///
/// tokio::run(future);
/// # }
/// ```
fn deadline(self, deadline: Instant) -> Deadline<Self>
where Self: Sized,
{
Deadline::new(self, deadline)
}
}
impl<T: ?Sized> FutureExt for T where T: Future {}
-9
View File
@@ -1,9 +0,0 @@
//! Utilities for working with Tokio.
//!
//! This module contains utilities that are useful for working with Tokio.
//! Currently, this only includes [`FutureExt`][FutureExt]. However, this will
//! include over time.
mod future;
pub use self::future::FutureExt;
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "tests-build"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[features]
full = ["tokio/full"]
[dependencies]
tokio = { path = "../tokio", optional = true }
[dev-dependencies]
trybuild = "1.0"
+2
View File
@@ -0,0 +1,2 @@
Tests the various combination of feature flags. This is broken out to a separate
crate to work around limitations with cargo features.
+2
View File
@@ -0,0 +1,2 @@
#[cfg(feature = "tokio")]
pub use tokio;
@@ -0,0 +1,25 @@
use tests_build::tokio;
#[tokio::main]
fn main_is_not_async() {}
#[tokio::main(foo)]
async fn main_attr_has_unknown_args() {}
#[tokio::main(threadpool::bar)]
async fn main_attr_has_path_args() {}
#[tokio::test]
fn test_is_not_async() {}
#[tokio::test]
async fn test_fn_has_args(_x: u8) {}
#[tokio::test(foo)]
async fn test_attr_has_args() {}
#[tokio::test]
#[test]
async fn test_has_second_test_attr() {}
fn main() {}
@@ -0,0 +1,41 @@
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:4:1
|
4 | fn main_is_not_async() {}
| ^^
error: Unknown attribute foo is specified; expected `basic_scheduler` or `threaded_scheduler`
--> $DIR/macros_invalid_input.rs:6:15
|
6 | #[tokio::main(foo)]
| ^^^
error: Must have specified ident
--> $DIR/macros_invalid_input.rs:9:15
|
9 | #[tokio::main(threadpool::bar)]
| ^^^^^^^^^^^^^^^
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:13:1
|
13 | fn test_is_not_async() {}
| ^^
error: the test function cannot accept arguments
--> $DIR/macros_invalid_input.rs:16:27
|
16 | async fn test_fn_has_args(_x: u8) {}
| ^^^^^^
error: Unknown attribute foo is specified; expected `basic_scheduler` or `threaded_scheduler`
--> $DIR/macros_invalid_input.rs:18:15
|
18 | #[tokio::test(foo)]
| ^^^
error: second test attribute is supplied
--> $DIR/macros_invalid_input.rs:22:1
|
22 | #[test]
| ^^^^^^^
+9
View File
@@ -0,0 +1,9 @@
#[test]
fn compile_fail() {
let t = trybuild::TestCases::new();
#[cfg(feature = "full")]
t.compile_fail("tests/fail/macros_invalid_input.rs");
drop(t);
}
+15
View File
@@ -0,0 +1,15 @@
[package]
name = "tests-integration"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[dependencies]
tokio = { path = "../tokio", features = ["full"] }
doc-comment = "0.3.1"
[dev-dependencies]
tokio-test = { path = "../tokio-test" }
futures = { version = "0.3.0", features = ["async-await"] }
+1
View File
@@ -0,0 +1 @@
Tests that require additional components than just the `tokio` crate.
+20
View File
@@ -0,0 +1,20 @@
//! A cat-like utility that can be used as a subprocess to test I/O
//! stream communication.
use std::io;
use std::io::Write;
fn main() {
let stdin = io::stdin();
let mut stdout = io::stdout();
let mut line = String::new();
loop {
line.clear();
stdin.read_line(&mut line).unwrap();
if line.is_empty() {
break;
}
stdout.write_all(line.as_bytes()).unwrap();
}
stdout.flush().unwrap();
}
+4
View File
@@ -0,0 +1,4 @@
use doc_comment::doc_comment;
// #[doc = include_str!("../../README.md")]
doc_comment!(include_str!("../../README.md"));
+126
View File
@@ -0,0 +1,126 @@
#![warn(rust_2018_idioms)]
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::process::{Child, Command};
use tokio_test::assert_ok;
use futures::future::{self, FutureExt};
use std::env;
use std::io;
use std::process::{ExitStatus, Stdio};
fn cat() -> Command {
let mut me = env::current_exe().unwrap();
me.pop();
if me.ends_with("deps") {
me.pop();
}
me.push("test-cat");
let mut cmd = Command::new(me);
cmd.stdin(Stdio::piped()).stdout(Stdio::piped());
cmd
}
async fn feed_cat(mut cat: Child, n: usize) -> io::Result<ExitStatus> {
let mut stdin = cat.stdin().take().unwrap();
let stdout = cat.stdout().take().unwrap();
// Produce n lines on the child's stdout.
let write = async {
for i in 0..n {
let bytes = format!("line {}\n", i).into_bytes();
stdin.write_all(&bytes).await.unwrap();
}
drop(stdin);
};
let read = async {
let mut reader = BufReader::new(stdout).lines();
let mut num_lines = 0;
// Try to read `n + 1` lines, ensuring the last one is empty
// (i.e. EOF is reached after `n` lines.
loop {
let data = reader
.next_line()
.await
.unwrap_or_else(|_| Some(String::new()))
.expect("failed to read line");
let num_read = data.len();
let done = num_lines >= n;
match (done, num_read) {
(false, 0) => panic!("broken pipe"),
(true, n) if n != 0 => panic!("extraneous data"),
_ => {
let expected = format!("line {}", num_lines);
assert_eq!(expected, data);
}
};
num_lines += 1;
if num_lines >= n {
break;
}
}
};
// Compose reading and writing concurrently.
future::join3(write, read, cat)
.map(|(_, _, status)| status)
.await
}
/// Check for the following properties when feeding stdin and
/// consuming stdout of a cat-like process:
///
/// - A number of lines that amounts to a number of bytes exceeding a
/// typical OS buffer size can be fed to the child without
/// deadlock. This tests that we also consume the stdout
/// concurrently; otherwise this would deadlock.
///
/// - We read the same lines from the child that we fed it.
///
/// - The child does produce EOF on stdout after the last line.
#[tokio::test]
async fn feed_a_lot() {
let child = cat().spawn().unwrap();
let status = feed_cat(child, 10000).await.unwrap();
assert_eq!(status.code(), Some(0));
}
#[tokio::test]
async fn wait_with_output_captures() {
let mut child = cat().spawn().unwrap();
let mut stdin = child.stdin().take().unwrap();
let write_bytes = b"1234";
let future = async {
stdin.write_all(write_bytes).await?;
drop(stdin);
let out = child.wait_with_output();
out.await
};
let output = future.await.unwrap();
assert!(output.status.success());
assert_eq!(output.stdout, write_bytes);
assert_eq!(output.stderr.len(), 0);
}
#[tokio::test]
async fn status_closes_any_pipes() {
// Cat will open a pipe between the parent and child.
// If `status_async` doesn't ensure the handles are closed,
// we would end up blocking forever (and time out).
let child = cat().status();
assert_ok!(child.await);
}
-63
View File
@@ -1,63 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::net::TcpStream;
use std::thread;
use std::io::{Read, Write, BufReader, BufWriter};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::copy;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
const N: usize = 1024;
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let t2 = thread::spawn(move || {
let mut s = t!(TcpStream::connect(&addr));
let mut b = vec![0; msg.len() * N];
t!(s.read_exact(&mut b));
b
});
let mut expected = Vec::<u8>::new();
for _i in 0..N {
expected.extend(msg.as_bytes());
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
}
(expected, t2)
});
let clients = srv.incoming().take(2).collect();
let copied = clients.and_then(|clients| {
let mut clients = clients.into_iter();
let a = BufReader::new(clients.next().unwrap());
let b = BufWriter::new(clients.next().unwrap());
copy(a, b)
});
let (amt, _, _) = t!(copied.wait());
let (expected, t2) = t.join().unwrap();
let actual = t2.join().unwrap();
assert!(expected == actual);
assert_eq!(amt, msg.len() as u64 * 1024);
}
-69
View File
@@ -1,69 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_timer;
extern crate env_logger;
use tokio::prelude::*;
use tokio::runtime::{self, current_thread};
use tokio::timer::*;
use tokio_timer::clock::Clock;
use std::sync::mpsc;
use std::time::{Duration, Instant};
struct MockNow(Instant);
impl tokio_timer::clock::Now for MockNow {
fn now(&self) -> Instant {
self.0
}
}
#[test]
fn clock_and_timer_concurrent() {
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = runtime::Builder::new()
.clock(clock)
.build()
.unwrap();
let (tx, rx) = mpsc::channel();
rt.spawn({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn clock_and_timer_single_threaded() {
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = current_thread::Builder::new()
.clock(clock)
.build()
.unwrap();
rt.block_on({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
Ok(())
})
}).unwrap();
}
-622
View File
@@ -1,622 +0,0 @@
#![cfg(not(feature = "unstable-futures"))]
extern crate tokio;
extern crate tokio_executor;
extern crate futures;
use tokio::executor::current_thread::{self, block_on_all, CurrentThread};
use std::any::Any;
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::prelude::*;
use futures::sync::oneshot;
#[test]
fn spawn_from_block_on_all() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let msg = current_thread::block_on_all(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Rc::new(Cell::new(0));
let cnt2 = cnt.clone();
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
assert_eq!(1, cnt2.get());
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
for _ in 0..ITER {
let cnt = cnt.clone();
current_thread.spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok::<(), ()>(())
}));
}
current_thread.run().unwrap();
assert_eq!(cnt.get(), ITER);
}
#[test]
fn does_not_set_global_executor_by_default() {
use tokio_executor::Executor;
block_on_all(lazy(|| {
tokio_executor::DefaultExecutor::current()
.spawn(Box::new(lazy(|| ok())))
.unwrap_err();
ok()
})).unwrap();
}
#[test]
fn spawn_from_block_on_future() {
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
current_thread::spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
}));
Ok::<_, ()>(())
})).unwrap();
current_thread.run().unwrap();
assert_eq!(1, cnt.get());
}
struct Never(Rc<()>);
impl Future for Never {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(Async::NotReady)
}
}
#[test]
fn outstanding_tasks_are_dropped_when_executor_is_dropped() {
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.spawn(Never(rc.clone()));
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
// Using the global spawn fn
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
current_thread::spawn(Never(rc.clone()));
Ok::<_, ()>(())
})).unwrap();
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
}
#[test]
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
})).unwrap();
}
#[test]
#[should_panic]
fn run_in_future() {
block_on_all(lazy(|| {
current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
}));
ok()
})).unwrap();
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
struct Infini {
num: Rc<Cell<usize>>,
}
impl Future for Infini {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.num.set(1 + self.num.get());
task::current().notify();
Ok(Async::NotReady)
}
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.turn(None)
.unwrap();
assert_eq!(1, num.get());
}
#[test]
fn tasks_are_scheduled_fairly() {
let state = Rc::new(RefCell::new([0, 0]));
struct Spin {
state: Rc<RefCell<[i32; 2]>>,
idx: usize,
}
impl Future for Spin {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
let mut state = self.state.borrow_mut();
if self.idx == 0 {
let diff = state[0] - state[1];
assert!(diff.abs() <= 1);
if state[0] >= 50 {
return Ok(().into());
}
}
state[self.idx] += 1;
if state[self.idx] >= 100 {
return Ok(().into());
}
task::current().notify();
Ok(Async::NotReady)
}
}
block_on_all(lazy(|| {
current_thread::spawn(Spin {
state: state.clone(),
idx: 0,
});
current_thread::spawn(Spin {
state: state,
idx: 1,
});
ok()
})).unwrap();
}
#[test]
fn spawn_and_turn() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let mut current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
current_thread.spawn(lazy(move || {
Ok(())
}));
// Turn once...
current_thread.turn(None).unwrap();
current_thread.spawn(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok(())
}));
// This does not run the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(1, cnt.get());
// This runs the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(2, cnt.get());
}
#[test]
fn spawn_in_drop() {
let mut current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
current_thread.spawn({
struct OnDrop<F: FnOnce()>(Option<F>);
impl<F: FnOnce()> Drop for OnDrop<F> {
fn drop(&mut self) {
(self.0.take().unwrap())();
}
}
struct MyFuture {
_data: Box<Any>,
}
impl Future for MyFuture {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(().into())
}
}
MyFuture {
_data: Box::new(OnDrop(Some(move || {
current_thread::spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}));
}))),
}
});
current_thread.block_on(rx).unwrap();
current_thread.run().unwrap();
}
#[test]
fn hammer_turn() {
use futures::sync::mpsc;
const ITER: usize = 100;
const N: usize = 100;
const THREADS: usize = 4;
for _ in 0..ITER {
let mut ths = vec![];
// Add some jitter
for _ in 0..THREADS {
let th = thread::spawn(|| {
let mut current_thread = CurrentThread::new();
let (tx, rx) = mpsc::unbounded();
current_thread.spawn({
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
rx.for_each(move |_| {
c.set(1 + c.get());
Ok(())
})
.map_err(|e| panic!("err={:?}", e))
.map(move |v| {
assert_eq!(N, cnt.get());
v
})
});
thread::spawn(move || {
for _ in 0..N {
tx.unbounded_send(()).unwrap();
thread::yield_now();
}
});
while !current_thread.is_idle() {
current_thread.turn(None).unwrap();
}
});
ths.push(th);
}
for th in ths {
th.join().unwrap();
}
}
}
#[test]
fn turn_has_polled() {
let mut current_thread = CurrentThread::new();
// Spawn oneshot receiver
let (sender, receiver) = oneshot::channel::<()>();
current_thread.spawn(receiver.then(|_| Ok(())));
// Turn once...
let res = current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// Should've polled the receiver once, but considered it not ready
assert!(res.has_polled());
// Turn another time
let res = current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// Should've polled nothing, the receiver is not ready yet
assert!(!res.has_polled());
// Make the receiver ready
sender.send(()).unwrap();
// Turn another time
let res = current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// Should've polled the receiver, it's ready now
assert!(res.has_polled());
// Now the executor should be empty
assert!(current_thread.is_idle());
let res = current_thread.turn(Some(Duration::from_millis(0))).unwrap();
// So should've polled nothing
assert!(!res.has_polled());
}
// Our own mock Park that is never really waiting and the only
// thing it does is to send, on request, something (once) to a onshot
// channel
struct MyPark {
sender: Option<oneshot::Sender<()>>,
send_now: Rc<Cell<bool>>,
}
struct MyUnpark;
impl tokio_executor::park::Park for MyPark {
type Unpark = MyUnpark;
type Error = ();
fn unpark(&self) -> Self::Unpark {
MyUnpark
}
fn park(&mut self) -> Result<(), Self::Error> {
// If called twice with send_now, this will intentionally panic
if self.send_now.get() {
self.sender.take().unwrap().send(()).unwrap();
}
Ok(())
}
fn park_timeout(&mut self, _duration: Duration) -> Result<(), Self::Error> {
self.park()
}
}
impl tokio_executor::park::Unpark for MyUnpark {
fn unpark(&self) {}
}
#[test]
fn turn_fair() {
let send_now = Rc::new(Cell::new(false));
let (sender, receiver) = oneshot::channel::<()>();
let (sender_2, receiver_2) = oneshot::channel::<()>();
let (sender_3, receiver_3) = oneshot::channel::<()>();
let my_park = MyPark {
sender: Some(sender_3),
send_now: send_now.clone(),
};
let mut current_thread = CurrentThread::new_with_park(my_park);
let receiver_1_done = Rc::new(Cell::new(false));
let receiver_1_done_clone = receiver_1_done.clone();
// Once an item is received on the oneshot channel, it will immediately
// immediately make the second oneshot channel ready
current_thread.spawn(receiver
.map_err(|_| unreachable!())
.and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
Ok(())
})
);
let receiver_2_done = Rc::new(Cell::new(false));
let receiver_2_done_clone = receiver_2_done.clone();
current_thread.spawn(receiver_2
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
})
);
// The third receiver is only woken up from our Park implementation, it simulates
// e.g. a socket that first has to be polled to know if it is ready now
let receiver_3_done = Rc::new(Cell::new(false));
let receiver_3_done_clone = receiver_3_done.clone();
current_thread.spawn(receiver_3
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
})
);
// First turn should've polled both and considered them not ready
let res = current_thread.turn(Some(Duration::from_millis(0))).unwrap();
assert!(res.has_polled());
// Next turn should've polled nothing
let res = current_thread.turn(Some(Duration::from_millis(0))).unwrap();
assert!(!res.has_polled());
assert!(!receiver_1_done.get());
assert!(!receiver_2_done.get());
assert!(!receiver_3_done.get());
// After this the receiver future will wake up the second receiver future,
// so there are pending futures again
sender.send(()).unwrap();
// Now the first receiver should be done, the second receiver should be ready
// to be polled again and the socket not yet
let res = current_thread.turn(None).unwrap();
assert!(res.has_polled());
assert!(receiver_1_done.get());
assert!(!receiver_2_done.get());
assert!(!receiver_3_done.get());
// Now let our park implementation know that it should send something to sender 3
send_now.set(true);
// This should resolve the second receiver directly, but also poll the socket
// and read the packet from it. If it didn't do both here, we would handle
// futures that are woken up from the reactor and directly unfairly and would
// favour the ones that are woken up directly.
let res = current_thread.turn(None).unwrap();
assert!(res.has_polled());
assert!(receiver_1_done.get());
assert!(receiver_2_done.get());
assert!(receiver_3_done.get());
// Don't send again
send_now.set(false);
// Now we should be idle and turning should not poll anything
assert!(current_thread.is_idle());
let res = current_thread.turn(None).unwrap();
assert!(!res.has_polled());
}
#[test]
fn spawn_from_other_thread() {
let mut current_thread = CurrentThread::new();
let handle = current_thread.handle();
let (sender, receiver) = oneshot::channel::<()>();
thread::spawn(move || {
handle.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
})).unwrap();
});
let _ = current_thread.block_on(receiver).unwrap();
}
#[test]
fn spawn_from_other_thread_unpark() {
use std::sync::mpsc::channel as mpsc_channel;
let mut current_thread = CurrentThread::new();
let handle = current_thread.handle();
let (sender_1, receiver_1) = oneshot::channel::<()>();
let (sender_2, receiver_2) = mpsc_channel::<()>();
thread::spawn(move || {
let _ = receiver_2.recv().unwrap();
handle.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
})).unwrap();
});
// Ensure that unparking the executor works correctly. It will first
// check if there are new futures (there are none), then execute the
// lazy future below which will cause the future to be spawned from
// the other thread. Then the executor will park but should be woken
// up because *now* we have a new future to schedule
let _ = current_thread.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1)
).unwrap();
}
fn ok() -> future::FutureResult<(), ()> {
future::ok(())
}
-42
View File
@@ -1,42 +0,0 @@
extern crate tokio;
extern crate futures;
use std::thread;
use std::net;
use futures::future;
use futures::prelude::*;
use futures::sync::oneshot;
use tokio::net::TcpListener;
use tokio::reactor::Reactor;
#[test]
fn tcp_doesnt_block() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
drop(core);
assert!(listener.incoming().wait().next().unwrap().is_err());
}
#[test]
fn drop_wakes() {
let core = Reactor::new().unwrap();
let handle = core.handle();
let listener = net::TcpListener::bind("127.0.0.1:0").unwrap();
let listener = TcpListener::from_std(listener, &handle).unwrap();
let (tx, rx) = oneshot::channel::<()>();
let t = thread::spawn(move || {
let incoming = listener.incoming();
let new_socket = incoming.into_future().map_err(|_| ());
let drop_tx = future::lazy(|| {
drop(tx);
future::ok(())
});
assert!(new_socket.join(drop_tx).wait().is_err());
});
drop(rx.wait());
drop(core);
t.join().unwrap();
}
-53
View File
@@ -1,53 +0,0 @@
#![cfg(feature = "unstable-futures")]
// This test is the same as `echo.rs`, but ported to futures 0.2
extern crate env_logger;
extern crate futures2;
extern crate tokio;
extern crate tokio_io;
use std::io::{Read, Write};
use std::net::TcpStream;
use std::thread;
use futures2::prelude::*;
use futures2::executor::block_on;
use tokio::net::TcpListener;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn echo_server() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let msg = "foo bar baz";
let t = thread::spawn(move || {
let mut s = TcpStream::connect(&addr).unwrap();
for _i in 0..1024 {
assert_eq!(t!(s.write(msg.as_bytes())), msg.len());
let mut buf = [0; 1024];
assert_eq!(t!(s.read(&mut buf)), msg.len());
assert_eq!(&buf[..msg.len()], msg.as_bytes());
}
});
let clients = srv.incoming();
let client = clients.next().map(|e| e.0.unwrap()).map_err(|e| e.0);
let halves = client.map(|s| s.split());
let copied = halves.and_then(|(a, b)| a.copy_into(b));
let (amt, _, _) = t!(block_on(copied));
t.join().unwrap();
assert_eq!(amt, msg.len() as u64 * 1024);
}
-136
View File
@@ -1,136 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn hammer_old() {
let _ = env_logger::init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().into_future()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(mine.join(theirs).wait());
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
}).collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
}
struct Rd(Arc<TcpStream>);
struct Wr(Arc<TcpStream>);
impl io::Read for Rd {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
<&TcpStream>::read(&mut &*self.0, dst)
}
}
impl tokio_io::AsyncRead for Rd {
}
impl io::Write for Wr {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
<&TcpStream>::write(&mut &*self.0, src)
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
impl tokio_io::AsyncWrite for Wr {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(().into())
}
}
#[test]
fn hammer_split() {
use tokio_io::io;
const N: usize = 100;
const ITER: usize = 10;
let _ = env_logger::init();
for _ in 0..ITER {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let cnt = Arc::new(AtomicUsize::new(0));
let mut rt = Runtime::new().unwrap();
fn split(socket: TcpStream, cnt: Arc<AtomicUsize>) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let cnt2 = cnt.clone();
let rd = io::read(rd, vec![0; 1])
.map(move |_| {
cnt2.fetch_add(1, Relaxed);
})
.map_err(|e| panic!("read error = {:?}", e));
let wr = io::write_all(wr, b"1")
.map(move |_| {
cnt.fetch_add(1, Relaxed);
})
.map_err(move |e| panic!("write error = {:?}", e));
tokio::spawn(rd);
tokio::spawn(wr);
}
rt.spawn({
let cnt = cnt.clone();
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(move |socket| {
split(socket, cnt.clone());
Ok(())
})
});
for _ in 0..N {
rt.spawn({
let cnt = cnt.clone();
TcpStream::connect(&addr)
.map_err(move |e| panic!("connect error = {:?}", e))
.map(move |socket| split(socket, cnt))
});
}
rt.shutdown_on_idle().wait().unwrap();
assert_eq!(N * 4, cnt.load(Relaxed));
}
}
-122
View File
@@ -1,122 +0,0 @@
#![cfg(feature = "unstable-futures")]
// This test is the same as `global.rs`, but ported to futures 0.2
extern crate futures;
extern crate futures2;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::Arc;
use futures2::prelude::*;
use futures2::executor::block_on;
use futures2::task;
use tokio::net::{TcpStream, TcpListener};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn hammer() {
let _ = env_logger::init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().next()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(block_on(mine.join(theirs)));
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
}).collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
}
struct Rd(Arc<TcpStream>);
struct Wr(Arc<TcpStream>);
impl AsyncRead for Rd {
fn poll_read(&mut self, cx: &mut task::Context, dst: &mut [u8]) -> Poll<usize, io::Error> {
<&TcpStream>::poll_read(&mut &*self.0, cx, dst)
}
}
impl AsyncWrite for Wr {
fn poll_write(&mut self, cx: &mut task::Context, src: &[u8]) -> Poll<usize, io::Error> {
<&TcpStream>::poll_write(&mut &*self.0, cx, src)
}
fn poll_flush(&mut self, _cx: &mut task::Context) -> Poll<(), io::Error> {
Ok(().into())
}
fn poll_close(&mut self, _cx: &mut task::Context) -> Poll<(), io::Error> {
Ok(().into())
}
}
#[test]
fn hammer_split() {
const N: usize = 100;
let _ = env_logger::init();
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mut rt = Runtime::new().unwrap();
fn split(socket: TcpStream) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let rd = rd.read(vec![0; 1])
.map(|_| ())
.map_err(|e| panic!("read error = {:?}", e));
let wr = wr.write_all(b"1")
.map(|_| ())
.map_err(|e| panic!("write error = {:?}", e));
tokio::spawn2(rd);
tokio::spawn2(wr);
}
rt.spawn2({
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(|socket| {
split(socket);
Ok(())
})
.map(|_| ())
});
for _ in 0..N {
rt.spawn2({
TcpStream::connect(&addr)
.map_err(|e| panic!("connect error = {:?}", e))
.map(|socket| split(socket))
});
}
futures::Future::wait(rt.shutdown_on_idle()).unwrap();
}
-88
View File
@@ -1,88 +0,0 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate tokio_threadpool;
extern crate bytes;
use std::io;
use std::net::Shutdown;
use bytes::{BytesMut, BufMut};
use futures::{Future, Stream, Sink};
use tokio::net::{TcpListener, TcpStream};
use tokio_codec::{Encoder, Decoder};
use tokio_io::io::{write_all, read};
use tokio_threadpool::Builder;
pub struct LineCodec;
impl Decoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<BytesMut>, io::Error> {
match buf.iter().position(|&b| b == b'\n') {
Some(i) => Ok(Some(buf.split_to(i + 1).into())),
None => Ok(None),
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> io::Result<Option<BytesMut>> {
if buf.len() == 0 {
Ok(None)
} else {
let amt = buf.len();
Ok(Some(buf.split_to(amt)))
}
}
}
impl Encoder for LineCodec {
type Item = BytesMut;
type Error = io::Error;
fn encode(&mut self, item: BytesMut, into: &mut BytesMut) -> io::Result<()> {
into.put(&item[..]);
Ok(())
}
}
#[test]
fn echo() {
drop(env_logger::init());
let pool = Builder::new()
.pool_size(1)
.build();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
let addr = listener.local_addr().unwrap();
let sender = pool.sender().clone();
let srv = listener.incoming().for_each(move |socket| {
let (sink, stream) = LineCodec.framed(socket).split();
sender.spawn(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
Ok(())
});
pool.sender().spawn(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
let client = TcpStream::connect(&addr);
let client = client.wait().unwrap();
let (client, _) = write_all(client, b"a\n").wait().unwrap();
let (client, buf, amt) = read(client, vec![0; 1024]).wait().unwrap();
assert_eq!(amt, 2);
assert_eq!(&buf[..2], b"a\n");
let (client, _) = write_all(client, b"\n").wait().unwrap();
let (client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"\n");
let (client, _) = write_all(client, b"b").wait().unwrap();
client.shutdown(Shutdown::Write).unwrap();
let (_client, buf, amt) = read(client, buf).wait().unwrap();
assert_eq!(amt, 1);
assert_eq!(&buf[..1], b"b");
}
-88
View File
@@ -1,88 +0,0 @@
#![cfg(unix)]
extern crate env_logger;
extern crate futures;
extern crate libc;
extern crate mio;
extern crate tokio;
extern crate tokio_io;
use std::fs::File;
use std::io::{self, Write};
use std::os::unix::io::{AsRawFd, FromRawFd};
use std::thread;
use std::time::Duration;
use mio::event::Evented;
use mio::unix::{UnixReady, EventedFd};
use mio::{PollOpt, Ready, Token};
use tokio::reactor::{Handle, PollEvented2};
use tokio_io::io::read_to_end;
use futures::Future;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
struct MyFile(File);
impl MyFile {
fn new(file: File) -> MyFile {
unsafe {
let r = libc::fcntl(file.as_raw_fd(), libc::F_SETFL, libc::O_NONBLOCK);
assert!(r != -1, "fcntl error: {}", io::Error::last_os_error());
}
MyFile(file)
}
}
impl io::Read for MyFile {
fn read(&mut self, bytes: &mut [u8]) -> io::Result<usize> {
self.0.read(bytes)
}
}
impl Evented for MyFile {
fn register(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).register(poll, token, interest | hup, opts)
}
fn reregister(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).reregister(poll, token, interest | hup, opts)
}
fn deregister(&self, poll: &mio::Poll) -> io::Result<()> {
EventedFd(&self.0.as_raw_fd()).deregister(poll)
}
}
#[test]
fn hup() {
drop(env_logger::init());
let handle = Handle::default();
unsafe {
let mut pipes = [0; 2];
assert!(libc::pipe(pipes.as_mut_ptr()) != -1,
"pipe error: {}", io::Error::last_os_error());
let read = File::from_raw_fd(pipes[0]);
let mut write = File::from_raw_fd(pipes[1]);
let t = thread::spawn(move || {
write.write_all(b"Hello!\n").unwrap();
write.write_all(b"Good bye!\n").unwrap();
thread::sleep(Duration::from_millis(100));
});
let source = PollEvented2::new_with_handle(MyFile::new(read), &handle).unwrap();
let reader = read_to_end(source, Vec::new());
let (_, content) = t!(reader.wait());
assert_eq!(&b"Hello!\nGood bye!\n"[..], &content[..]);
t.join().unwrap();
}
}
-175
View File
@@ -1,175 +0,0 @@
extern crate tokio;
extern crate env_logger;
extern crate futures;
use futures::sync::oneshot;
use std::sync::{Arc, Mutex};
use std::thread;
use tokio::io;
use tokio::net::{TcpStream, TcpListener};
use tokio::prelude::future::lazy;
use tokio::prelude::*;
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
fn create_client_server_future() -> Box<Future<Item=(), Error=()> + Send> {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| panic!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| ())
.map_err(|e| panic!("write err = {:?}", e))
})
})
.map(|_| ());
let client = client
.map_err(|e| panic!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| ())
.map_err(|e| panic!("read err = {:?}", e))
});
let future = server.join(client)
.map(|_| ());
Box::new(future)
}
#[test]
fn runtime_tokio_run() {
let _ = env_logger::init();
tokio::run(create_client_server_future());
}
#[test]
fn runtime_single_threaded() {
let _ = env_logger::init();
let mut runtime = tokio::runtime::current_thread::Runtime::new()
.unwrap();
runtime.block_on(create_client_server_future()).unwrap();
runtime.run().unwrap();
}
#[test]
fn runtime_multi_threaded() {
let _ = env_logger::init();
let mut runtime = tokio::runtime::Builder::new()
.build()
.unwrap();
runtime.spawn(create_client_server_future());
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn block_on_timer() {
use std::time::{Duration, Instant};
use tokio::timer::{Delay, Error};
fn after_1s<T>(x: T) -> Box<Future<Item = T, Error = Error> + Send>
where
T: Send + 'static,
{
Box::new(Delay::new(Instant::now() + Duration::from_millis(100)).map(move |_| x))
}
let mut runtime = Runtime::new().unwrap();
assert_eq!(runtime.block_on(after_1s(42)).unwrap(), 42);
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn_from_block_on() {
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
let msg = runtime
.block_on(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
tokio::spawn(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
}))
.unwrap();
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
use std::time::Duration;
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
runtime
.block_on(rx.then(move |_| {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<_, ()>(())
}))
.unwrap();
assert_eq!(1, *cnt.lock().unwrap());
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Arc::new(Mutex::new(0));
let mut runtime = Runtime::new().unwrap();
for _ in 0..ITER {
let c = cnt.clone();
runtime.spawn(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
}));
}
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(ITER, *cnt.lock().unwrap());
}
-136
View File
@@ -1,136 +0,0 @@
#![cfg(feature = "unstable-futures")]
// This test is the same as `tcp.rs`, but ported to futures 0.2
extern crate env_logger;
extern crate tokio;
extern crate mio;
extern crate futures2;
use std::{net, thread};
use std::sync::mpsc::channel;
use tokio::net::{TcpListener, TcpStream};
use futures2::executor::block_on;
use futures2::prelude::*;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
}
#[test]
fn connect() {
drop(env_logger::init());
let srv = t!(net::TcpListener::bind("127.0.0.1:0"));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
t!(srv.accept()).0
});
let stream = TcpStream::connect(&addr);
let mine = t!(block_on(stream));
let theirs = t.join().unwrap();
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
}
#[test]
fn accept() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let (tx, rx) = channel();
let client = srv.incoming().map(move |t| {
tx.send(()).unwrap();
t
}).next().map_err(|e| e.0);
assert!(rx.try_recv().is_err());
let t = thread::spawn(move || {
net::TcpStream::connect(&addr).unwrap()
});
let (mine, _remaining) = t!(block_on(client));
let mine = mine.unwrap();
let theirs = t.join().unwrap();
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
}
#[test]
fn accept2() {
drop(env_logger::init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
net::TcpStream::connect(&addr).unwrap()
});
let (tx, rx) = channel();
let client = srv.incoming().map(move |t| {
tx.send(()).unwrap();
t
}).next().map_err(|e| e.0);
assert!(rx.try_recv().is_err());
let (mine, _remaining) = t!(block_on(client));
mine.unwrap();
t.join().unwrap();
}
#[cfg(unix)]
mod unix {
use tokio::net::TcpStream;
use tokio::prelude::*;
use env_logger;
use futures2::future;
use futures2::executor::block_on;
use futures2::io::AsyncRead;
use mio::unix::UnixReady;
use std::{net, thread};
use std::time::Duration;
#[test]
fn poll_hup() {
drop(env_logger::init());
let srv = t!(net::TcpListener::bind("127.0.0.1:0"));
let addr = t!(srv.local_addr());
let t = thread::spawn(move || {
let mut client = t!(srv.accept()).0;
client.write(b"hello world").unwrap();
thread::sleep(Duration::from_millis(200));
});
let mut stream = t!(block_on(TcpStream::connect(&addr)));
// Poll for HUP before reading.
block_on(future::poll_fn(|cx| {
stream.poll_read_ready2(cx, UnixReady::hup().into())
})).unwrap();
// Same for write half
block_on(future::poll_fn(|cx| {
stream.poll_write_ready2(cx)
})).unwrap();
let mut buf = vec![0; 11];
// Read the data
block_on(future::poll_fn(|cx| {
stream.poll_read(cx, &mut buf)
})).unwrap();
assert_eq!(b"hello world", &buf[..]);
t.join().unwrap();
}
}
-94
View File
@@ -1,94 +0,0 @@
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use tokio::prelude::*;
use tokio::timer::*;
use std::sync::mpsc;
use std::time::{Duration, Instant};
#[test]
fn timer_with_runtime() {
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(100);
let (tx, rx) = mpsc::channel();
tokio::run({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn starving() {
use futures::{task, Poll, Async};
let _ = env_logger::init();
struct Starve(Delay, u64);
impl Future for Starve {
type Item = u64;
type Error = ();
fn poll(&mut self) -> Poll<Self::Item, ()> {
if self.0.poll().unwrap().is_ready() {
return Ok(self.1.into());
}
self.1 += 1;
task::current().notify();
Ok(Async::NotReady)
}
}
let when = Instant::now() + Duration::from_millis(20);
let starve = Starve(Delay::new(when), 0);
let (tx, rx) = mpsc::channel();
tokio::run({
starve
.and_then(move |_ticks| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn deadline() {
use futures::future;
let _ = env_logger::init();
let when = Instant::now() + Duration::from_millis(20);
let (tx, rx) = mpsc::channel();
tokio::run({
future::empty::<(), ()>()
.deadline(when)
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
-3
View File
@@ -1,3 +0,0 @@
# Unreleased
* Initial release (#353)
-22
View File
@@ -1,22 +0,0 @@
[package]
name = "tokio-codec"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>", "Bryan Burgers <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-codec/0.1"
description = """
Utilities for encoding and decoding frames.
"""
categories = ["asynchronous"]
[dependencies]
tokio-io = { version = "0.1.6", path = "../tokio-io" }
bytes = "0.4.7"
futures = "0.1.18"
-35
View File
@@ -1,35 +0,0 @@
# tokio-codec
Utilities for encoding and decoding frames.
[Documentation](https://docs.rs/tokio-codec)
## Usage
First, add this to your `Cargo.toml`:
```toml
[dependencies]
tokio-codec = "0.1"
```
Next, add this to your crate:
```rust
extern crate tokio_codec;
```
You can find extensive documentation and examples about how to use this crate
online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-codec) is also a great place to get started
for the nitty-gritty.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-32
View File
@@ -1,32 +0,0 @@
//! Utilities for encoding and decoding frames.
//!
//! Contains adapters to go from streams of bytes, [`AsyncRead`] and
//! [`AsyncWrite`], to framed streams implementing [`Sink`] and [`Stream`].
//! Framed streams are also known as [transports].
//!
//! [`AsyncRead`]: #
//! [`AsyncWrite`]: #
//! [`Sink`]: #
//! [`Stream`]: #
//! [transports]: #
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-codec/0.1.0")]
extern crate bytes;
extern crate tokio_io;
mod bytes_codec;
mod lines_codec;
pub use tokio_io::_tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
};
pub use bytes_codec::BytesCodec;
pub use lines_codec::LinesCodec;
-89
View File
@@ -1,89 +0,0 @@
use bytes::{BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::{io, str};
/// A simple `Codec` implementation that splits up data into lines.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
pub struct LinesCodec {
// Stored index of the next index to examine for a `\n` character.
// This is used to optimize searching.
// For example, if `decode` was called with `abc`, it would hold `3`,
// because that is the next index to examine.
// The next time `decode` is called with `abcde\n`, the method will
// only look at `de\n` before returning.
next_index: usize,
}
impl LinesCodec {
/// Returns a `LinesCodec` for splitting up data into lines.
pub fn new() -> LinesCodec {
LinesCodec { next_index: 0 }
}
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf).map_err(|_|
io::Error::new(
io::ErrorKind::InvalidData,
"Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
if let Some(&b'\r') = s.last() {
&s[..s.len() - 1]
} else {
s
}
}
impl Decoder for LinesCodec {
type Item = String;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<String>, io::Error> {
if let Some(newline_offset) =
buf[self.next_index..].iter().position(|b| *b == b'\n')
{
let newline_index = newline_offset + self.next_index;
let line = buf.split_to(newline_index + 1);
let line = &line[..line.len()-1];
let line = without_carriage_return(line);
let line = utf8(line)?;
self.next_index = 0;
Ok(Some(line.to_string()))
} else {
self.next_index = buf.len();
Ok(None)
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> Result<Option<String>, io::Error> {
Ok(match self.decode(buf)? {
Some(frame) => Some(frame),
None => {
// No terminating newline - return remaining data, if any
if buf.is_empty() || buf == &b"\r"[..] {
None
} else {
let line = buf.take();
let line = without_carriage_return(&line);
let line = utf8(line)?;
self.next_index = 0;
Some(line.to_string())
}
}
})
}
}
impl Encoder for LinesCodec {
type Item = String;
type Error = io::Error;
fn encode(&mut self, line: String, buf: &mut BytesMut) -> Result<(), io::Error> {
buf.reserve(line.len() + 1);
buf.put(line);
buf.put_u8(b'\n');
Ok(())
}
}
-76
View File
@@ -1,76 +0,0 @@
extern crate tokio_codec;
extern crate bytes;
use bytes::{BytesMut, Bytes, BufMut};
use tokio_codec::{BytesCodec, LinesCodec, Decoder, Encoder};
#[test]
fn bytes_decoder() {
let mut codec = BytesCodec::new();
let buf = &mut BytesMut::new();
buf.put_slice(b"abc");
assert_eq!("abc", codec.decode(buf).unwrap().unwrap());
assert_eq!(None, codec.decode(buf).unwrap());
assert_eq!(None, codec.decode(buf).unwrap());
buf.put_slice(b"a");
assert_eq!("a", codec.decode(buf).unwrap().unwrap());
}
#[test]
fn bytes_encoder() {
let mut codec = BytesCodec::new();
// Default capacity of BytesMut
#[cfg(target_pointer_width = "64")]
const INLINE_CAP: usize = 4 * 8 - 1;
#[cfg(target_pointer_width = "32")]
const INLINE_CAP: usize = 4 * 4 - 1;
let mut buf = BytesMut::new();
codec.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf).unwrap();
// Default capacity of Framed Read
const INITIAL_CAPACITY: usize = 8 * 1024;
let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);
codec.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf).unwrap();
}
#[test]
fn lines_decoder() {
let mut codec = LinesCodec::new();
let buf = &mut BytesMut::new();
buf.reserve(200);
buf.put("line 1\nline 2\r\nline 3\n\r\n\r");
assert_eq!("line 1", codec.decode(buf).unwrap().unwrap());
assert_eq!("line 2", codec.decode(buf).unwrap().unwrap());
assert_eq!("line 3", codec.decode(buf).unwrap().unwrap());
assert_eq!("", codec.decode(buf).unwrap().unwrap());
assert_eq!(None, codec.decode(buf).unwrap());
assert_eq!(None, codec.decode_eof(buf).unwrap());
buf.put("k");
assert_eq!(None, codec.decode(buf).unwrap());
assert_eq!("\rk", codec.decode_eof(buf).unwrap().unwrap());
assert_eq!(None, codec.decode(buf).unwrap());
assert_eq!(None, codec.decode_eof(buf).unwrap());
}
#[test]
fn lines_encoder() {
let mut codec = BytesCodec::new();
// Default capacity of BytesMut
#[cfg(target_pointer_width = "64")]
const INLINE_CAP: usize = 4 * 8 - 1;
#[cfg(target_pointer_width = "32")]
const INLINE_CAP: usize = 4 * 4 - 1;
let mut buf = BytesMut::new();
codec.encode(Bytes::from_static(&[b'a'; INLINE_CAP + 1]), &mut buf).unwrap();
// Default capacity of Framed Read
const INITIAL_CAPACITY: usize = 8 * 1024;
let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);
codec.encode(Bytes::from_static(&[b'a'; INITIAL_CAPACITY + 1]), &mut buf).unwrap();
}
-216
View File
@@ -1,216 +0,0 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
use tokio_io::AsyncRead;
use tokio_codec::{FramedRead, Decoder};
use bytes::{BytesMut, Buf, IntoBuf, BigEndian};
use futures::Stream;
use futures::Async::{Ready, NotReady};
use std::io::{self, Read};
use std::collections::VecDeque;
macro_rules! mock {
($($x:expr,)*) => {{
let mut v = VecDeque::new();
v.extend(vec![$($x),*]);
Mock { calls: v }
}};
}
struct U32Decoder;
impl Decoder for U32Decoder {
type Item = u32;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<u32>> {
if buf.len() < 4 {
return Ok(None);
}
let n = buf.split_to(4).into_buf().get_u32::<BigEndian>();
Ok(Some(n))
}
}
#[test]
fn read_multi_frame_in_packet() {
let mock = mock! {
Ok(b"\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02".to_vec()),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert_eq!(Ready(Some(1)), framed.poll().unwrap());
assert_eq!(Ready(Some(2)), framed.poll().unwrap());
assert_eq!(Ready(None), framed.poll().unwrap());
}
#[test]
fn read_multi_frame_across_packets() {
let mock = mock! {
Ok(b"\x00\x00\x00\x00".to_vec()),
Ok(b"\x00\x00\x00\x01".to_vec()),
Ok(b"\x00\x00\x00\x02".to_vec()),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert_eq!(Ready(Some(1)), framed.poll().unwrap());
assert_eq!(Ready(Some(2)), framed.poll().unwrap());
assert_eq!(Ready(None), framed.poll().unwrap());
}
#[test]
fn read_not_ready() {
let mock = mock! {
Err(io::Error::new(io::ErrorKind::WouldBlock, "")),
Ok(b"\x00\x00\x00\x00".to_vec()),
Ok(b"\x00\x00\x00\x01".to_vec()),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(NotReady, framed.poll().unwrap());
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert_eq!(Ready(Some(1)), framed.poll().unwrap());
assert_eq!(Ready(None), framed.poll().unwrap());
}
#[test]
fn read_partial_then_not_ready() {
let mock = mock! {
Ok(b"\x00\x00".to_vec()),
Err(io::Error::new(io::ErrorKind::WouldBlock, "")),
Ok(b"\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02".to_vec()),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(NotReady, framed.poll().unwrap());
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert_eq!(Ready(Some(1)), framed.poll().unwrap());
assert_eq!(Ready(Some(2)), framed.poll().unwrap());
assert_eq!(Ready(None), framed.poll().unwrap());
}
#[test]
fn read_err() {
let mock = mock! {
Err(io::Error::new(io::ErrorKind::Other, "")),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(io::ErrorKind::Other, framed.poll().unwrap_err().kind());
}
#[test]
fn read_partial_then_err() {
let mock = mock! {
Ok(b"\x00\x00".to_vec()),
Err(io::Error::new(io::ErrorKind::Other, "")),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(io::ErrorKind::Other, framed.poll().unwrap_err().kind());
}
#[test]
fn read_partial_would_block_then_err() {
let mock = mock! {
Ok(b"\x00\x00".to_vec()),
Err(io::Error::new(io::ErrorKind::WouldBlock, "")),
Err(io::Error::new(io::ErrorKind::Other, "")),
};
let mut framed = FramedRead::new(mock, U32Decoder);
assert_eq!(NotReady, framed.poll().unwrap());
assert_eq!(io::ErrorKind::Other, framed.poll().unwrap_err().kind());
}
#[test]
fn huge_size() {
let data = [0; 32 * 1024];
let mut framed = FramedRead::new(&data[..], BigDecoder);
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert_eq!(Ready(None), framed.poll().unwrap());
struct BigDecoder;
impl Decoder for BigDecoder {
type Item = u32;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> io::Result<Option<u32>> {
if buf.len() < 32 * 1024 {
return Ok(None);
}
buf.split_to(32 * 1024);
Ok(Some(0))
}
}
}
#[test]
fn data_remaining_is_error() {
let data = [0; 5];
let mut framed = FramedRead::new(&data[..], U32Decoder);
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert!(framed.poll().is_err());
}
#[test]
fn multi_frames_on_eof() {
struct MyDecoder(Vec<u32>);
impl Decoder for MyDecoder {
type Item = u32;
type Error = io::Error;
fn decode(&mut self, _buf: &mut BytesMut) -> io::Result<Option<u32>> {
unreachable!();
}
fn decode_eof(&mut self, _buf: &mut BytesMut) -> io::Result<Option<u32>> {
if self.0.is_empty() {
return Ok(None);
}
Ok(Some(self.0.remove(0)))
}
}
let mut framed = FramedRead::new(mock!(), MyDecoder(vec![0, 1, 2, 3]));
assert_eq!(Ready(Some(0)), framed.poll().unwrap());
assert_eq!(Ready(Some(1)), framed.poll().unwrap());
assert_eq!(Ready(Some(2)), framed.poll().unwrap());
assert_eq!(Ready(Some(3)), framed.poll().unwrap());
assert_eq!(Ready(None), framed.poll().unwrap());
}
// ===== Mock ======
struct Mock {
calls: VecDeque<io::Result<Vec<u8>>>,
}
impl Read for Mock {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(data)) => {
debug_assert!(dst.len() >= data.len());
dst[..data.len()].copy_from_slice(&data[..]);
Ok(data.len())
}
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
}
impl AsyncRead for Mock {
}

Some files were not shown because too many files have changed in this diff Show More