Compare commits

..
30 Commits
Author SHA1 Message Date
Alex Crichton 571a48255e Bump to 0.1.2 2017-01-24 11:02:36 -08:00
Alex Crichton ba19515833 Clarify a comment 2017-01-24 10:59:31 -08:00
Alex Crichton a21153a0e9 Clear out old Signal on drop 2017-01-24 10:54:55 -08:00
Alex Crichton 83a5277af1 Touch up more impls and comments 2017-01-24 10:47:09 -08:00
Alex Crichton b748fa955c Handle a few more errors 2017-01-24 10:06:54 -08:00
Alex Crichton 277fff43f5 Various cleanups:
* Drop nix/lazy_static
* Use previously registered handlers
* Handle some more errors
2017-01-24 10:02:42 -08:00
Michal 'vorner' Vaner a02a38fbae The driver task
Add the driver task, connecting the signal handler wakeups to the
wakeups of of the streams.

It is a prototype-quality code, a lot of cleanups and similar is needed.
2017-01-22 14:48:22 +01:00
Michal 'vorner' Vaner f7167b48a7 Provide the new kind of Signal stream
Which is just a wrapper around the futures::sync::mpsc. The sender is in
a global registry.

The part that connects the wakeups to the senders in the registry
doesn't yet exist.
2017-01-22 12:58:48 +01:00
Michal 'vorner' Vaner 8abc26df39 Implement the wake-up part of the new signal handling
Register the signal handler that wakes up someone through a self-pipe.
That someone doesn't yet exist, though.

Some dependencies (nix, lazy_static) added to speed up the prototyping
process. They are likely to be dropped in some future commits.

Some features (eg. preserving the previous signal handlers) are still
missing.
2017-01-22 12:27:03 +01:00
Michal 'vorner' Vaner b9da819153 Use tokio-core from git
Just for now, as we need some yet unreleased features.
2017-01-22 10:19:38 +01:00
Alex Crichton 10151bfca0 Update docs urls and such 2017-01-12 10:32:16 -08:00
Alex Crichton a3c07eb771 Remove deprecated API usage on Windows 2017-01-12 10:29:22 -08:00
Alex Crichton 18776138ad Remove use of deprecated APIs on Unix 2017-01-12 10:25:51 -08:00
Alex Crichton cf9398ef61 Bump to 0.1.1 2017-01-11 18:56:15 -08:00
Alex Crichton b726610e7f Ignore errors in signal handler
Closes #3
2017-01-11 10:21:01 -08:00
Alex Crichton 338567ca80 Update travis token 2016-11-19 09:15:38 -08:00
Alex Crichton 3181ebfda6 Merge pull request #2 from jugglerchris/patch-1
Trivial typo fix.
2016-11-07 14:25:06 -08:00
Chris Emerson 424be889a4 Trivial typo fix. 2016-11-07 22:04:08 +00:00
Alex Crichton 72283f178a Remove SIGKILL reexport 2016-10-05 13:55:45 -07:00
Alex Crichton 202220034e Add symbolic reexports for common signals
Means you don't have to import libc!

Closes #1
2016-10-05 08:58:06 -07:00
Alex Crichton 0c0c0bb177 Update deps to point to crates.io 2016-09-09 22:04:04 -07:00
Alex Crichton 69ced1b6da Start adding windows support 2016-09-08 17:28:44 -07:00
Alex Crichton 17481a3a54 Track tokio-core master 2016-09-07 22:14:54 -07:00
Alex Crichton 93e7d9759a Fix travis token 2016-09-07 00:14:23 -07:00
Alex Crichton 85d6d43d18 Update travis link 2016-09-07 00:06:14 -07:00
Alex Crichton e74b728ecf Update Cargo metadata 2016-09-07 00:05:45 -07:00
Alex Crichton 9acd70b7e3 Add licenses 2016-09-07 00:04:49 -07:00
Alex Crichton a531721b1c Add a README 2016-09-07 00:04:44 -07:00
Alex Crichton 06153d0f28 Add docs and travis 2016-09-07 00:03:43 -07:00
Alex Crichton 50973e0734 Initial commit 2016-09-06 23:00:17 -07:00
514 changed files with 1100 additions and 84243 deletions
-29
View File
@@ -1,29 +0,0 @@
freebsd_instance:
image: freebsd-12-2-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD
env:
LOOM_MAX_PREEMPTIONS: 2
RUSTFLAGS: -Dwarnings
setup_script:
- pkg install -y bash curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y --profile minimal --default-toolchain stable
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
test_script:
- . $HOME/.cargo/env
- cargo test --all
- cargo doc --all --no-deps
i686_test_script:
- . $HOME/.cargo/env
- |
cargo test --all --target i686-unknown-freebsd
-36
View File
@@ -1,36 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: A-tokio, C-bug
assignees: ''
---
**Version**
List the versions of all `tokio` crates you are using. The easiest way to get
this information is using `cargo-tree`.
`cargo install cargo-tree`
(see install here: https://github.com/sfackler/cargo-tree)
Then:
`cargo tree | grep tokio`
**Platform**
The output of `uname -a` (UNIX), or version and 32 or 64-bit (Windows)
**Description**
Enter your issue details here.
One way to structure the description:
[short summary of the bug]
I tried this code:
[code sample that causes the bug]
I expected to see this happen: [explanation]
Instead, this happened: [explanation]
-20
View File
@@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: A-tokio, C-feature-request
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
-16
View File
@@ -1,16 +0,0 @@
---
name: Question
about: Please use the discussions tab for questions
title: ''
labels: ''
assignees: ''
---
Please post your question as a discussion here:
https://github.com/tokio-rs/tokio/discussions
You may also be able to find help here:
https://discord.gg/tokio
https://users.rust-lang.org/
-26
View File
@@ -1,26 +0,0 @@
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.
Bug fixes and new features should include tests.
Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
The contributors guide includes instructions for running rustfmt and building the
documentation, which requires special commands beyond `cargo fmt` and `cargo doc`.
-->
## Motivation
<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->
## Solution
<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
-22
View File
@@ -1,22 +0,0 @@
name: Security Audit
on:
push:
branches:
- master
paths:
- '**/Cargo.toml'
schedule:
- cron: '0 2 * * *' # run at 2 AM UTC
jobs:
security-audit:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, 'ci skip')"
steps:
- uses: actions/checkout@v2
- name: Audit Check
uses: actions-rs/audit-check@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
-55
View File
@@ -1,55 +0,0 @@
name: Benchmark
on:
push:
branches:
- master
jobs:
benchmark:
name: Benchmark
runs-on: ubuntu-latest
strategy:
matrix:
bench:
- rt_multi_threaded
- sync_mpsc
- sync_rwlock
- sync_semaphore
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable
# Run benchmark with `go test -bench` and stores the output to a file
- name: Run benchmark
run: cargo bench --bench ${{ matrix.bench }} | tee ../output.txt
working-directory: benches
# Download previous benchmark result from cache (if exists)
- name: Download previous benchmark data
uses: actions/cache@v1
with:
path: ./cache
key: ${{ runner.os }}-benchmark
# Run `github-action-benchmark` action
- name: Store benchmark result
uses: rhysd/github-action-benchmark@v1
with:
name: ${{ matrix.bench }}
# What benchmark tool the output.txt came from
tool: 'cargo'
# Where the output from the benchmark tool is stored
output-file-path: output.txt
# # Where the previous data file is stored
# external-data-json-path: ./cache/benchmark-data.json
# Workflow will fail when an alert happens
fail-on-alert: true
# GitHub API token to make a commit comment
github-token: ${{ secrets.GITHUB_TOKEN }}
# Enable alert commit comment
comment-on-alert: true
alert-comment-cc-users: '@tokio-rs/maintainers'
auto-push: true
# Upload the updated cache file for the next job by actions/cache
-275
View File
@@ -1,275 +0,0 @@
on:
push:
branches: ["v0.3.x"]
pull_request:
branches: ["v0.3.x"]
name: CI
env:
RUSTFLAGS: -Dwarnings
RUST_BACKTRACE: 1
nightly: nightly-2020-09-21
minrust: 1.45.2
jobs:
# Depends on all action sthat are required for a "successful" CI run.
tests-pass:
name: all systems go
runs-on: ubuntu-latest
needs:
- test
- test-unstable
- miri
- cross
- features
- minrust
- fmt
- clippy
- docs
- loom
steps:
- run: exit 0
test:
name: test tokio full
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- windows-latest
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable
- name: Install cargo-hack
run: cargo install cargo-hack
# Run `tokio` with `full` features. This excludes testing utilities which
# can alter the runtime behavior of Tokio.
- name: test tokio full
run: cargo test --features full
working-directory: tokio
# Test **all** crates in the workspace with all features.
- name: test all --all-features
run: cargo test --workspace --all-features
# Run integration tests for each feature
- name: test tests-integration --each-feature
run: cargo hack test --each-feature
working-directory: tests-integration
# Run macro build tests
- name: test tests-build --each-feature
run: cargo hack test --each-feature
working-directory: tests-build
test-unstable:
name: test tokio full --unstable
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- windows-latest
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable
# Run `tokio` with "unstable" cfg flag.
- name: test tokio full --cfg unstable
run: cargo test --features full
working-directory: tokio
env:
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
miri:
name: miri
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.nightly }}
override: true
- name: Install Miri
run: |
set -e
rustup component add miri
cargo miri setup
rm -rf tokio/tests
- name: miri
run: cargo miri test --features rt,rt-multi-thread,sync task
working-directory: tokio
san:
name: san
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.nightly }}
override: true
- name: asan
run: cargo test --all-features --target x86_64-unknown-linux-gnu --lib -- --test-threads 1
working-directory: tokio
env:
RUSTFLAGS: -Z sanitizer=address
ASAN_OPTIONS: detect_leaks=0
cross:
name: cross
runs-on: ubuntu-latest
strategy:
matrix:
target:
- i686-unknown-linux-gnu
- powerpc-unknown-linux-gnu
- powerpc64-unknown-linux-gnu
- mips-unknown-linux-gnu
- arm-linux-androideabi
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
target: ${{ matrix.target }}
override: true
- uses: actions-rs/cargo@v1
with:
use-cross: true
command: check
args: --workspace --target ${{ matrix.target }}
features:
name: features
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.nightly }}
override: true
- name: Install cargo-hack
run: cargo install cargo-hack
- name: check --each-feature
run: cargo hack check --all --each-feature -Z avoid-dev-deps
# Try with unstable feature flags
- name: check --each-feature --unstable
run: cargo hack check --all --each-feature -Z avoid-dev-deps
env:
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
minrust:
name: minrust
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.minrust }}
override: true
- name: "test --workspace --all-features"
run: cargo check --workspace --all-features
minimal-versions:
name: minimal-versions
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.nightly }}
override: true
- name: Install cargo-hack
run: cargo install cargo-hack
- name: "check --all-features -Z minimal-versions"
run: |
# Remove dev-dependencies from Cargo.toml to prevent the next `cargo update`
# from determining minimal versions based on dev-dependencies.
cargo hack --remove-dev-deps --workspace
# Update Cargo.lock to minimal version dependencies.
cargo update -Z minimal-versions
cargo check --all-features
fmt:
name: fmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable
- name: Install rustfmt
run: rustup component add rustfmt
# Check fmt
- name: "rustfmt --check"
# Workaround for rust-lang/cargo#7732
run: |
if ! rustfmt --check --edition 2018 $(find . -name '*.rs' -print); then
printf "Please run \`rustfmt --edition 2018 \$(find . -name '*.rs' -print)\` to fix rustfmt errors.\nSee CONTRIBUTING.md for more details.\n" >&2
exit 1
fi
clippy:
name: clippy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update ${{ env.minrust }} && rustup default ${{ env.minrust }}
- name: Install clippy
run: rustup component add clippy
# Run clippy
- name: "clippy --all"
run: cargo clippy --all --tests
docs:
name: docs
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.nightly }}
override: true
- name: "doc --lib --all-features"
run: cargo doc --lib --no-deps --all-features
env:
RUSTDOCFLAGS: --cfg docsrs
loom:
name: loom
runs-on: ubuntu-latest
strategy:
matrix:
scope:
- --skip loom_pool
- loom_pool::group_a
- loom_pool::group_b
- loom_pool::group_c
- loom_pool::group_d
- time::driver
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable
- name: loom ${{ matrix.scope }}
run: cargo test --lib --release --features full -- --nocapture $SCOPE
working-directory: tokio
env:
RUSTFLAGS: --cfg loom --cfg tokio_unstable -Dwarnings
LOOM_MAX_PREEMPTIONS: 2
SCOPE: ${{ matrix.scope }}
-32
View File
@@ -1,32 +0,0 @@
name: Pull Request Security Audit
on:
push:
paths:
- '**/Cargo.toml'
pull_request:
paths:
- '**/Cargo.toml'
jobs:
security-audit:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, 'ci skip')"
steps:
- uses: actions/checkout@v2
- name: Install cargo-audit
uses: actions-rs/cargo@v1
with:
command: install
args: cargo-audit
- name: Generate lockfile
uses: actions-rs/cargo@v1
with:
command: generate-lockfile
- name: Audit dependencies
uses: actions-rs/cargo@v1
with:
command: audit
-32
View File
@@ -1,32 +0,0 @@
name: Stress Test
on:
pull_request:
push:
branches:
- master
jobs:
stess-test:
name: Stress Test
runs-on: ubuntu-latest
strategy:
matrix:
stress-test:
- simple_echo_tcp
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable
- name: Install Valgrind
run: |
sudo apt-get update -y
sudo apt-get install -y valgrind
# Compiles each of the stress test examples.
- name: Compile stress test examples
run: cargo build -p stress-test --release --example ${{ matrix.stress-test }}
# Runs each of the examples using Valgrind. Detects leaks and displays them.
- name: Run valgrind
run: valgrind --leak-check=full --show-leak-kinds=all ./target/release/examples/${{ matrix.stress-test }}
+25
View File
@@ -0,0 +1,25 @@
language: rust
rust:
- stable
- beta
- nightly
sudo: false
before_script:
- pip install 'travis-cargo<0.2' --user && export PATH=$HOME/.local/bin:$PATH
script:
- cargo build
- cargo test
- cargo doc --no-deps
after_success:
- travis-cargo --only nightly doc-upload
env:
global:
- secure: "SXXK7Znvm1s5WWQ94l9IP25mXA0uGIQ7ghBuumZz3nSAfxhhJQnYi5hCAbl2/cOfSbpgEtE137dgk6Nd9UDx7rIkLCSe8TWyYjzraX/vvX3xNtLh/fjsayYYRK9a6qU2HIJegZdxPgyF5h2DeBgeLks0Ue8drrFQ1s9bYZVUO0yeuZ3aLkL1FkIG6RXGItUFpb6srEYL1NLizYLxXFEG3cL+kKoFIWc2qPx3EwOqv/eii134nQsuObhWZvPqfTo7zfNP8W/6TnoiggpRH1nrZc3DI3CynTICIOJ2Ogn9gFX9LftYKuJysSwUNVN3WF5aOuLP/XjRSBLYc+PW3v0iqiGzMX3n1VpcyhcbsSNA7ZckGn1HZsWYwspAxkN3idSuVie9Mezm7IV4005juiYKEWEr6hlkv1lzd49QZkWOvLCFCMRiwOOGp4NyzilG1Q1Zs3G1wrcvstmasNpK+QUFNdOFvT2sm34rI4x2rQUvjC/OyqbAK+PjYmTHL47YKON5ymfUL3mAcwgUfBUSd4Wpx8G3VKg3gMcmQm27ah1knOGJWH6XulYTnfGfx6bLo5t2NGx+vZk0naqajD3auWnseobMDsFjhUIRrt6GlnfPqeFoJSm0unu3riAX+RDF/iqZdDfjhX4evETIw3SaTl8EQtVLwz7kJTnxSbTU4XTi+0M="
notifications:
email:
on_success: never
os:
- linux
- osx
-7
View File
@@ -1,7 +0,0 @@
# Code of Conduct
The Tokio project adheres to the [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct). This describes the minimum behavior expected from all contributors.
## Enforcement
Instances of violations of the Code of Conduct can be reported by contacting the project team at [[email protected]](mailto:[email protected]).
-563
View File
@@ -1,563 +0,0 @@
# Contributing to Tokio
:balloon: Thanks for your help improving the project! We are so happy to have
you!
There are opportunities to contribute to Tokio at any level. It doesn't matter if
you are just getting started with Rust or are the most weathered expert, we can
use your help.
**No contribution is too small and all contributions are valued.**
This guide will help you get started. **Do not let this guide intimidate you**.
It should be considered a map to help you navigate the process.
The [dev channel][dev] is available for any concerns not covered in this guide, please join
us!
[dev]: https://discord.gg/tokio
## Conduct
The Tokio project adheres to the [Rust Code of Conduct][coc]. This describes
the _minimum_ behavior expected from all contributors. Instances of violations of the
Code of Conduct can be reported by contacting the project team at
[[email protected]](mailto:[email protected]).
[coc]: https://github.com/rust-lang/rust/blob/master/CODE_OF_CONDUCT.md
## Contributing in Issues
For any issue, there are fundamentally three ways an individual can contribute:
1. By opening the issue for discussion: For instance, if you believe that you
have discovered a bug in Tokio, creating a new issue in [the tokio-rs/tokio
issue tracker][issue] is the way to report it.
2. By helping to triage the issue: This can be done by providing
supporting details (a test case that demonstrates a bug), providing
suggestions on how to address the issue, or ensuring that the issue is tagged
correctly.
3. By helping to resolve the issue: Typically this is done either in the form of
demonstrating that the issue reported is not a problem after all, or more
often, by opening a Pull Request that changes some bit of something in
Tokio in a concrete and reviewable manner.
[issue]: https://github.com/tokio-rs/tokio/issues
**Anybody can participate in any stage of contribution**. We urge you to
participate in the discussion around bugs and participate in reviewing PRs.
### Asking for General Help
If you have reviewed existing documentation and still have questions or are
having problems, you can [open a discussion] asking for help.
In exchange for receiving help, we ask that you contribute back a documentation
PR that helps others avoid the problems that you encountered.
[open a discussion]: https://github.com/tokio-rs/tokio/discussions/new
### Submitting a Bug Report
When opening a new issue in the Tokio issue tracker, you will be presented
with a basic template that should be filled in. If you believe that you have
uncovered a bug, please fill out this form, following the template to the best
of your ability. Do not worry if you cannot answer every detail, just fill in
what you can.
The two most important pieces of information we need in order to properly
evaluate the report is a description of the behavior you are seeing and a simple
test case we can use to recreate the problem on our own. If we cannot recreate
the issue, it becomes impossible for us to fix.
In order to rule out the possibility of bugs introduced by userland code, test
cases should be limited, as much as possible, to using only Tokio APIs.
See [How to create a Minimal, Complete, and Verifiable example][mcve].
[mcve]: https://stackoverflow.com/help/mcve
### Triaging a Bug Report
Once an issue has been opened, it is not uncommon for there to be discussion
around it. Some contributors may have differing opinions about the issue,
including whether the behavior being seen is a bug or a feature. This discussion
is part of the process and should be kept focused, helpful, and professional.
Short, clipped responses—that provide neither additional context nor supporting
detail—are not helpful or professional. To many, such responses are simply
annoying and unfriendly.
Contributors are encouraged to help one another make forward progress as much as
possible, empowering one another to solve issues collaboratively. If you choose
to comment on an issue that you feel either is not a problem that needs to be
fixed, or if you encounter information in an issue that you feel is incorrect,
explain why you feel that way with additional supporting context, and be willing
to be convinced that you may be wrong. By doing so, we can often reach the
correct outcome much faster.
### Resolving a Bug Report
In the majority of cases, issues are resolved by opening a Pull Request. The
process for opening and reviewing a Pull Request is similar to that of opening
and triaging issues, but carries with it a necessary review and approval
workflow that ensures that the proposed changes meet the minimal quality and
functional guidelines of the Tokio project.
## Pull Requests
Pull Requests are the way concrete changes are made to the code, documentation,
and dependencies in the Tokio repository.
Even tiny pull requests (e.g., one character pull request fixing a typo in API
documentation) are greatly appreciated. Before making a large change, it is
usually a good idea to first open an issue describing the change to solicit
feedback and guidance. This will increase the likelihood of the PR getting
merged.
### Cargo Commands
Due to the extensive use of features in Tokio, you will often need to add extra
arguments to many common cargo commands. This section lists some commonly needed
commands.
Some commands just need the `--all-features` argument:
```
cargo build --all-features
cargo check --all-features
cargo test --all-features
```
When building documentation normally, the markers that list the features
required for various parts of Tokio are missing. To build the documentation
correctly, use this command:
```
RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc --all-features
```
The `cargo fmt` command does not work on the Tokio codebase. You can use the
command below instead:
```
# Mac or Linux
rustfmt --check --edition 2018 $(find . -name '*.rs' -print)
# Powershell
Get-ChildItem . -Filter "*.rs" -Recurse | foreach { rustfmt --check --edition 2018 $_.FullName }
```
The `--check` argument prints the things that need to be fixed. If you remove
it, `rustfmt` will update your files locally instead.
You can run loom tests with
```
cd tokio # tokio crate in workspace
LOOM_MAX_PREEMPTIONS=1 RUSTFLAGS="--cfg loom" \
cargo test --lib --release --features full -- --test-threads=1 --nocapture
```
### Tests
If the change being proposed alters code (as opposed to only documentation for
example), it is either adding new functionality to Tokio or it is fixing
existing, broken functionality. In both of these cases, the pull request should
include one or more tests to ensure that Tokio does not regress in the future.
There are two ways to write tests: integration tests and documentation tests
(Tokio avoids unit tests as much as possible).
#### Integration tests
Integration tests go in the same crate as the code they are testing. Each sub
crate should have a `dev-dependency` on `tokio` itself. This makes all Tokio
utilities available to use in tests, no matter the crate being tested.
The best strategy for writing a new integration test is to look at existing
integration tests in the crate and follow the style.
#### Documentation tests
Ideally, every API has at least one [documentation test] that demonstrates how to
use the API. Documentation tests are run with `cargo test --doc`. This ensures
that the example is correct and provides additional test coverage.
The trick to documentation tests is striking a balance between being succinct
for a reader to understand and actually testing the API.
Same as with integration tests, when writing a documentation test, the full
`tokio` crate is available. This is especially useful for getting access to the
runtime to run the example.
The documentation tests will be visible from both the crate specific
documentation **and** the `tokio` facade documentation via the re-export. The
example should be written from the point of view of a user that is using the
`tokio` crate. As such, the example should use the API via the facade and not by
directly referencing the crate.
The type level example for `tokio_timer::Timeout` provides a good example of a
documentation test:
```
/// // import the `timeout` function, usually this is done
/// // with `use tokio::prelude::*`
/// use tokio::prelude::FutureExt;
/// use futures::Stream;
/// use futures::sync::mpsc;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = mpsc::unbounded();
/// # tx.unbounded_send(()).unwrap();
/// # drop(tx);
///
/// let process = rx.for_each(|item| {
/// // do something with `item`
/// # drop(item);
/// # Ok(())
/// });
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// process.timeout(Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
Given that this is a *type* level documentation test and the primary way users
of `tokio` will create an instance of `Timeout` is by using
`FutureExt::timeout`, this is how the documentation test is structured.
Lines that start with `/// #` are removed when the documentation is generated.
They are only there to get the test to run. The `block_on_all` function is the
easiest way to execute a future from a test.
If this were a documentation test for the `Timeout::new` function, then the
example would explicitly use `Timeout::new`. For example:
```
/// use tokio::timer::Timeout;
/// use futures::Future;
/// use futures::sync::oneshot;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = oneshot::channel();
/// # tx.send(()).unwrap();
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// Timeout::new(rx, Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
### Commits
It is a recommended best practice to keep your changes as logically grouped as
possible within individual commits. There is no limit to the number of commits
any single Pull Request may have, and many contributors find it easier to review
changes that are split across multiple commits.
That said, if you have a number of commits that are "checkpoints" and don't
represent a single logical change, please squash those together.
Note that multiple commits often get squashed when they are landed (see the
notes about [commit squashing](#commit-squashing)).
#### Commit message guidelines
A good commit message should describe what changed and why.
1. The first line should:
* contain a short description of the change (preferably 50 characters or less,
and no more than 72 characters)
* be entirely in lowercase with the exception of proper nouns, acronyms, and
the words that refer to code, like function/variable names
* be prefixed with the name of the sub crate being changed (without the `tokio-`
prefix) and start with an imperative verb. If modifying `tokio` proper,
omit the crate prefix.
Examples:
* timer: introduce `Timeout` and deprecate `Deadline`
* export `Encoder`, `Decoder`, `Framed*` from tokio_codec
2. Keep the second line blank.
3. Wrap all other lines at 72 columns (except for long URLs).
4. If your patch fixes an open issue, you can add a reference to it at the end
of the log. Use the `Fixes: #` prefix and the issue number. For other
references use `Refs: #`. `Refs` may include multiple issues, separated by a
comma.
Examples:
- `Fixes: #1337`
- `Refs: #1234`
Sample complete commit message:
```txt
subcrate: explain the commit in one line
Body of commit message is a few lines of text, explaining things
in more detail, possibly giving some background about the issue
being fixed, etc.
The body of the commit message can be several paragraphs, and
please do proper word-wrap and keep columns shorter than about
72 characters or so. That way, `git log` will show things
nicely even when it is indented.
Fixes: #1337
Refs: #453, #154
```
### Opening the Pull Request
From within GitHub, opening a new Pull Request will present you with a
[template] that should be filled out. Please try to do your best at filling out
the details, but feel free to skip parts if you're not sure what to put.
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Discuss and update
You will probably get feedback or requests for changes to your Pull Request.
This is a big part of the submission process so don't be discouraged! Some
contributors may sign off on the Pull Request right away, others may have
more detailed comments or feedback. This is a necessary part of the process
in order to evaluate whether the changes are correct and necessary.
**Any community member can review a PR and you might get conflicting feedback**.
Keep an eye out for comments from code owners to provide guidance on conflicting
feedback.
**Once the PR is open, do not rebase the commits**. See [Commit Squashing](#commit-squashing) for
more details.
### Commit Squashing
In most cases, **do not squash commits that you add to your Pull Request during
the review process**. When the commits in your Pull Request land, they may be
squashed into one commit per logical change. Metadata will be added to the
commit message (including links to the Pull Request, links to relevant issues,
and the names of the reviewers). The commit history of your Pull Request,
however, will stay intact on the Pull Request page.
## Reviewing Pull Requests
**Any Tokio community member is welcome to review any pull request**.
All Tokio contributors who choose to review and provide feedback on Pull
Requests have a responsibility to both the project and the individual making the
contribution. Reviews and feedback must be helpful, insightful, and geared
towards improving the contribution as opposed to simply blocking it. If there
are reasons why you feel the PR should not land, explain what those are. Do not
expect to be able to block a Pull Request from advancing simply because you say
"No" without giving an explanation. Be open to having your mind changed. Be open
to working with the contributor to make the Pull Request better.
Reviews that are dismissive or disrespectful of the contributor or any other
reviewers are strictly counter to the Code of Conduct.
When reviewing a Pull Request, the primary goals are for the codebase to improve
and for the person submitting the request to succeed. **Even if a Pull Request
does not land, the submitters should come away from the experience feeling like
their effort was not wasted or unappreciated**. Every Pull Request from a new
contributor is an opportunity to grow the community.
### Review a bit at a time.
Do not overwhelm new contributors.
It is tempting to micro-optimize and make everything about relative performance,
perfect grammar, or exact style matches. Do not succumb to that temptation.
Focus first on the most significant aspects of the change:
1. Does this change make sense for Tokio?
2. Does this change make Tokio better, even if only incrementally?
3. Are there clear bugs or larger scale issues that need attending to?
4. Is the commit message readable and correct? If it contains a breaking change
is it clear enough?
Note that only **incremental** improvement is needed to land a PR. This means
that the PR does not need to be perfect, only better than the status quo. Follow
up PRs may be opened to continue iterating.
When changes are necessary, *request* them, do not *demand* them, and **do not
assume that the submitter already knows how to add a test or run a benchmark**.
Specific performance optimization techniques, coding styles and conventions
change over time. The first impression you give to a new contributor never does.
Nits (requests for small changes that are not essential) are fine, but try to
avoid stalling the Pull Request. Most nits can typically be fixed by the Tokio
Collaborator landing the Pull Request but they can also be an opportunity for
the contributor to learn a bit more about the project.
It is always good to clearly indicate nits when you comment: e.g.
`Nit: change foo() to bar(). But this is not blocking.`
If your comments were addressed but were not folded automatically after new
commits or if they proved to be mistaken, please, [hide them][hiding-a-comment]
with the appropriate reason to keep the conversation flow concise and relevant.
### Be aware of the person behind the code
Be aware that *how* you communicate requests and reviews in your feedback can
have a significant impact on the success of the Pull Request. Yes, we may land
a particular change that makes Tokio better, but the individual might just not
want to have anything to do with Tokio ever again. The goal is not just having
good code.
### Abandoned or Stalled Pull Requests
If a Pull Request appears to be abandoned or stalled, it is polite to first
check with the contributor to see if they intend to continue the work before
checking if they would mind if you took it over (especially if it just has nits
left). When doing so, it is courteous to give the original contributor credit
for the work they started (either by preserving their name and email address in
the commit log, or by using an `Author: ` meta-data tag in the commit.
_Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
## Keeping track of issues and PRs
The Tokio GitHub repository has a lot of issues and PRs, which is not easy to
keep track of. This section explains the meaning of various labels, as well as
our [GitHub project][project]. The section is primarily targeted at maintainers.
**Area.** The area label describes the crates relevant to this issue or PR.
- **A-tokio** This issue concerns the main Tokio crate.
- **A-tokio-util** This issue concerns the `tokio-util` crate.
- **A-tokio-tls** This issue concerns the `tokio-tls` crate. Only used for
older issues, as the crate has been moved to another repository.
- **A-tokio-test** The issue concerns the `tokio-test` crate.
- **A-tokio-macros** This issue concerns the `tokio-macros` crate. Should only
be used for the procedural macros, and not `join!` or `select!`.
- **A-ci** This issue concerns our GitHub Actions setup.
**Category.** The category label describes the category.
- **C-bug** This is a bug-report. Bug-fix PRs use `C-enhancement` instead.
- **C-enhancement** This is a PR that adds a new features.
- **C-maintenance** This is an issue or PR about stuff such as documentation,
GitHub Actions or code quality.
- **C-feature-request** This is a feature request. Implementations of feature
requests use `C-enhancement` instead.
- **C-feature-accepted** If you submit a PR for this feature request, we wont
close it with the reason "we don't want this". Issues with this label should
also have the `C-feature-request` label.
- **C-musing** Stuff like tracking issues or roadmaps. "musings about a better
world"
- **C-proposal** A proposal of some kind, and a request for comments.
- **C-question** A user question. Large overlap with GitHub discussions.
- **C-request** A non-feature request, e.g. "please add deprecation notices to
`-alpha.*` versions of crates"
**Call for participation.** I don't know why it's called `E-`. Many issues are
missing a difficulty rating, and you should feel free to add one.
- **E-help-wanted** Stuff where we want help. Often seen together with `C-bug`
or `C-feature-accepted`.
- **E-easy** This is easy, ranging from quick documentation fixes to stuff you
can do after reading the tutorial on our website.
- **E-medium** This is not `E-easy` or `E-hard`.
- **E-hard** This either involves very tricky code, is something we don't know
how to solve, or is difficult for some other reason.
- **E-needs-mvce** This bug is missing a minimal complete and verifiable
example.
**Module.** A more fine groaned categorization than area.
- **M-blocking** Things relevant to `spawn_blocking`, `block_in_place`.
- **M-codec** The `tokio_util::codec` module.
- **M-compat** The `tokio_util::compat` module.
- **M-coop** Things relevant to coop.
- **M-fs** The `tokio::fs` module.
- **M-io** The `tokio::io` module.
- **M-macros** Issues about any kind of macro.
- **M-net** The `tokio::net` module.
- **M-process** The `tokio::process` module.
- **M-runtime** The `tokio::runtime` module.
- **M-signal** The `tokio::signal` module.
- **M-stream** The `tokio::stream` module.
- **M-sync** The `tokio::sync` module.
- **M-task** The `tokio::task` module.
- **M-time** The `tokio::time` module.
- **M-tracing** Tracing support in Tokio.
**Topic.** Some extra information.
- **T-docs** This is about documentation.
- **T-performance** This is about performance.
- **T-v0.1.x** This is about old Tokio.
Any label not listed here is not in active use.
[project]: https://github.com/orgs/tokio-rs/projects/1
## Releasing
Since the Tokio project consists of a number of crates, many of which depend on
each other, releasing new versions to crates.io can involve some complexities.
When releasing a new version of a crate, follow these steps:
1. **Ensure that the release crate has no path dependencies.** When the HEAD
version of a Tokio crate requires unreleased changes in another Tokio crate,
the crates.io dependency on the second crate will be replaced with a path
dependency. Crates with path dependencies cannot be published, so before
publishing the dependent crate, any path dependencies must also be published.
This should be done through a form of depth-first tree traversal:
1. Starting with the first path dependency in the crate to be released,
inspect the `Cargo.toml` for the dependency. If the dependency has any
path dependencies of its own, repeat this step with the first such
dependency.
2. Begin the release process for the path dependency.
3. Once the path dependency has been published to crates.io, update the
dependent crate to depend on the crates.io version.
4. When all path dependencies have been published, the dependent crate may
be published.
To verify that a crate is ready to publish, run:
```bash
bin/publish --dry-run <CRATE NAME> <CRATE VERSION>
```
2. **Update Cargo metadata.** After releasing any path dependencies, update the
`version` field in `Cargo.toml` to the new version, and the `documentation`
field to the docs.rs URL of the new version.
3. **Update other documentation links.** Update the `#![doc(html_root_url)]`
attribute in the crate's `lib.rs` and the "Documentation" link in the crate's
`README.md` to point to the docs.rs URL of the new version.
4. **Update the changelog for the crate.** Each crate in the Tokio repository
has its own `CHANGELOG.md` in that crate's subdirectory. Any changes to that
crate since the last release should be added to the changelog. Change
descriptions may be taken from the Git history, but should be edited to
ensure a consistent format, based on [Keep A Changelog][keep-a-changelog].
Other entries in that crate's changelog may also be used for reference.
5. **Perform a final audit for breaking changes.** Compare the HEAD version of
crate with the Git tag for the most recent release version. If there are any
breaking API changes, determine if those changes can be made without breaking
existing APIs. If so, resolve those issues. Otherwise, if it is necessary to
make a breaking release, update the version numbers to reflect this.
6. **Open a pull request with your changes.** Once that pull request has been
approved by a maintainer and the pull request has been merged, continue to
the next step.
7. **Release the crate.** Run the following command:
```bash
bin/publish <NAME OF CRATE> <VERSION>
```
Your editor and prompt you to edit a message for the tag. Copy the changelog
entry for that release version into your editor and close the window.
[keep-a-changelog]: https://github.com/olivierlacan/keep-a-changelog/blob/master/CHANGELOG.md
+23 -13
View File
@@ -1,15 +1,25 @@
[workspace]
[package]
name = "tokio-signal"
version = "0.1.2"
authors = ["Alex Crichton <[email protected]>"]
license = "MIT/Apache-2.0"
repository = "https://github.com/alexcrichton/tokio-signal"
homepage = "https://github.com/alexcrichton/tokio-signal"
documentation = "https://docs.rs/tokio-signal/0.1"
description = """
An implementation of an asynchronous Unix signal handling backed futures.
"""
members = [
"tokio",
"tokio-macros",
"tokio-test",
"tokio-util",
[dependencies]
tokio-core = "0.1.4"
futures = "0.1.7"
# Internal
"benches",
"examples",
"stress-test",
"tests-build",
"tests-integration",
]
[target.'cfg(unix)'.dependencies]
libc = "0.2"
mio = "0.6"
mio-uds = "0.6"
[target.'cfg(windows)'.dependencies]
winapi = "0.2"
kernel32-sys = "0.2"
mio = "0.6"
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2019 Tokio Contributors
Copyright (c) 2016 Alex Crichton
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
+18 -160
View File
@@ -1,174 +1,32 @@
# Tokio
# tokio-signal
A runtime for writing reliable, asynchronous, and slim applications with
the Rust programming language. It is:
An implementation of Unix signal handling for Tokio
* **Fast**: Tokio's zero-cost abstractions give you bare-metal
performance.
[![Build Status](https://travis-ci.org/alexcrichton/tokio-signal.svg?branch=master)](https://travis-ci.org/alexcrichton/tokio-signal)
* **Reliable**: Tokio leverages Rust's ownership, type system, and
concurrency model to reduce bugs and ensure thread safety.
[Documentation](https://docs.rs/tokio-signal)
* **Scalable**: Tokio has a minimal footprint, and handles backpressure
and cancellation naturally.
## Usage
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Build Status][actions-badge]][actions-url]
[![Discord chat][discord-badge]][discord-url]
First, add this to your `Cargo.toml`:
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: https://github.com/tokio-rs/tokio/blob/master/LICENSE
[actions-badge]: https://github.com/tokio-rs/tokio/workflows/CI/badge.svg
[actions-url]: https://github.com/tokio-rs/tokio/actions?query=workflow%3ACI+branch%3Amaster
[discord-badge]: https://img.shields.io/discord/500028886025895936.svg?logo=discord&style=flat-square
[discord-url]: https://discord.gg/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/tokio/tutorial) |
[API Docs](https://docs.rs/tokio/latest/tokio) |
[Roadmap](https://github.com/tokio-rs/tokio/blob/master/ROADMAP.md) |
[Chat](https://discord.gg/tokio)
## Overview
Tokio is an event-driven, non-blocking I/O platform for writing
asynchronous applications with the Rust programming language. At a high
level, it provides a few major components:
* A multithreaded, work-stealing based task [scheduler].
* A reactor backed by the operating system's event queue (epoll, kqueue,
IOCP, etc...).
* Asynchronous [TCP and UDP][net] sockets.
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/latest/tokio/net/index.html
[scheduler]: https://docs.rs/tokio/latest/tokio/runtime/index.html
## Example
A basic TCP echo server with Tokio:
```rust,no_run
use tokio::net::TcpListener;
use tokio::prelude::*;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut listener = TcpListener::bind("127.0.0.1:8080").await?;
loop {
let (mut socket, _) = listener.accept().await?;
tokio::spawn(async move {
let mut buf = [0; 1024];
// In a loop, read data from the socket and write the data back.
loop {
let n = match socket.read(&mut buf).await {
// socket closed
Ok(n) if n == 0 => return,
Ok(n) => n,
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
return;
}
};
// Write the data back
if let Err(e) = socket.write_all(&buf[0..n]).await {
eprintln!("failed to write to socket; err = {:?}", e);
return;
}
}
});
}
}
```toml
[dependencies]
tokio-signal = "0.1"
```
More examples can be found [here][examples]. For a larger "real world" example, see the
[mini-redis] repository.
Next, add this to your crate:
[examples]: https://github.com/tokio-rs/tokio/tree/master/examples
[mini-redis]: https://github.com/tokio-rs/mini-redis/
```rust
extern crate tokio_signal;
```
To see a list of the available features flags that can be enabled, check our
[docs][feature-flag-docs].
# License
## Getting Help
`tokio-signal` is primarily distributed under the terms of both the MIT
license and the Apache License (Version 2.0), with portions covered by various
BSD-like licenses.
First, see if the answer to your question can be found in the [Guides] or the
[API documentation]. If the answer is not there, there is an active community in
the [Tokio Discord server][chat]. We would be happy to try to answer your
question. You can also ask your question on [the discussions page][discussions].
See LICENSE-APACHE, and LICENSE-MIT for details.
[Guides]: https://tokio.rs/tokio/tutorial
[API documentation]: https://docs.rs/tokio/latest/tokio
[chat]: https://discord.gg/tokio
[discussions]: https://github.com/tokio-rs/tokio/discussions
[feature-flag-docs]: https://docs.rs/tokio/#feature-flags
## Contributing
:balloon: Thanks for your help improving the project! We are so happy to have
you! We have a [contributing guide][guide] to help you get involved in the Tokio
project.
[guide]: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
## Related Projects
In addition to the crates in this repository, the Tokio project also maintains
several other libraries, including:
* [`hyper`]: A fast and correct HTTP/1.1 and HTTP/2 implementation for Rust.
* [`tonic`]: A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility.
* [`warp`]: A super-easy, composable, web server framework for warp speeds.
* [`tower`]: A library of modular and reusable components for building robust networking clients and servers.
* [`tracing`] (formerly `tokio-trace`): A framework for application-level
tracing and async-aware diagnostics.
* [`rdbc`]: A Rust database connectivity library for MySQL, Postgres and SQLite.
* [`mio`]: A low-level, cross-platform abstraction over OS I/O APIs that powers
`tokio`.
* [`bytes`]: Utilities for working with bytes, including efficient byte buffers.
* [`loom`]: A testing tool for concurrent Rust code
[`warp`]: https://github.com/seanmonstar/warp
[`hyper`]: https://github.com/hyperium/hyper
[`tonic`]: https://github.com/hyperium/tonic
[`tower`]: https://github.com/tower-rs/tower
[`loom`]: https://github.com/tokio-rs/loom
[`rdbc`]: https://github.com/tokio-rs/rdbc
[`tracing`]: https://github.com/tokio-rs/tracing
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Supported Rust Versions
Tokio is built against the latest stable release. The minimum supported version is 1.45.
The current Tokio version is not guaranteed to build on Rust versions earlier than the
minimum supported version.
## License
This project is licensed under the [MIT license].
[MIT license]: https://github.com/tokio-rs/tokio/blob/master/LICENSE
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-67
View File
@@ -1,67 +0,0 @@
# Tokio Roadmap
## A Roadmap to 1.0
The question of "why not 1.0?" has come up a few times. After all, Tokio 0.1 has
been stable for three years. The short answer: because it isn't time. There is
nobody who would rather ship a Tokio 1.0 than us. It also isn't something to rush.
After all, `async / await` only landed in the stable Rust channel weeks ago.
There has been no significant production validation yet, except maybe fuchsia
and that seems like a fairly specialized use case. This release of Tokio
includes significant new code and new strategies with feature flags. Also, there
are still big open questions, such as the [proposed changes][pr-1744] to
`AsyncRead` and `AsyncWrite`.
Tokio 1.0 will be released as soon as the APIs are proven to handle real-world
production cases.
### Tokio 1.0 in Q3 2020 with LTS support
The Tokio 1.0 release will be **no later** than Q3 2020. It will also come with
"long-term support" guarantees:
* A minimum of 5 years of maintenance.
* A minimum of 3 years before a hypothetical 2.0 release.
When Tokio 1.0 is released in Q3 2020, on-going support, security fixes, and
critical bug fixes are guaranteed until **at least** Q3 2025. Tokio 2.0 will not
be released until **at least** Q3 2023 (though, ideally there will never be a
Tokio 2.0 release).
### How to get there
While Tokio 0.1 probably should have been a 1.0, Tokio 0.2 will be a **true**
0.2 release. There will be breaking change releases every 2 ~ 3 months until 1.0.
These changes will be **much** smaller than going from 0.1 -> 0.2. It is
expected that the 1.0 release will look a lot like 0.2.
### What is expected to change
The biggest change will be the `AsyncRead` and `AsyncWrite` traits. Based on
experience gained over the past 3 years, there are a couple of issues to
address:
* Be able to **safely** use uninitialized memory as a read buffer.
* Practical read vectored and write vectored APIs.
There are a few strategies to solve these problems. These strategies need to be
investigated and the solution validated. You can see [this comment][pr-1744-comment] for a
detailed statement of the problem.
The other major change, which has been in the works for a while, is updating
Mio. Mio 0.6 was first released almost 4 years ago and has not had a breaking
change since. Mio 0.7 has been in the works for a while. It includes a full
rewrite of the windows support as well as a refined API. More will be written
about this shortly.
Finally, now that the API is starting to stabilize, effort will be put into
documentation. Tokio 0.2 is being released before updating the website and many
of the old content will no longer be relevant. In the coming weeks, expect to
see updates there.
So, we have our work cut out for us. We hope you enjoy this 0.2 release and are
looking forward to your feedback and help.
[pr-1744]: https://github.com/tokio-rs/tokio/pull/1744
[pr-1744-comment]: https://github.com/tokio-rs/tokio/pull/1744#issuecomment-553575438
-13
View File
@@ -1,13 +0,0 @@
## Report a security issue
The Tokio project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via [[email protected]](mailto:[email protected]). Security issues should not be reported via the public Github Issue tracker.
## Vulnerability coordination
Remediation of security vulnerabilities is prioritized by the project team. The project team coordinates remediation with third-party project stakeholders via [Github Security Advisories](https://help.github.com/en/github/managing-security-vulnerabilities/about-github-security-advisories). Third-party stakeholders may include the reporter of the issue, affected direct or indirect users of Tokio, and maintainers of upstream dependencies if applicable.
Downstream project maintainers and Tokio users can request participation in coordination of applicable security issues by sending your contact email address, Github username(s) and any other salient information to [[email protected]](mailto:[email protected]). Participation in security issue coordination processes is at the discretion of the Tokio team.
## Security advisories
The project team is committed to transparency in the security issue disclosure process. The Tokio team announces security issues via [project Github Release notes](https://github.com/tokio-rs/tokio/releases) and the [RustSec advisory database](https://github.com/RustSec/advisory-db) (i.e. `cargo-audit`).
-43
View File
@@ -1,43 +0,0 @@
[package]
name = "benches"
version = "0.0.0"
publish = false
edition = "2018"
[dependencies]
tokio = { version = "0.3.0", path = "../tokio", features = ["full"] }
bencher = "0.1.5"
[target.'cfg(unix)'.dependencies]
libc = "0.2.42"
[[bench]]
name = "spawn"
path = "spawn.rs"
harness = false
[[bench]]
name = "sync_mpsc"
path = "sync_mpsc.rs"
harness = false
[[bench]]
name = "rt_multi_threaded"
path = "rt_multi_threaded.rs"
harness = false
[[bench]]
name = "sync_rwlock"
path = "sync_rwlock.rs"
harness = false
[[bench]]
name = "sync_semaphore"
path = "sync_semaphore.rs"
harness = false
[[bench]]
name = "signal"
path = "signal.rs"
harness = false
-151
View File
@@ -1,151 +0,0 @@
//! Benchmark implementation details of the theaded scheduler. These benches are
//! intended to be used as a form of regression testing and not as a general
//! purpose benchmark demonstrating real-world performance.
use tokio::runtime::{self, Runtime};
use tokio::sync::oneshot;
use bencher::{benchmark_group, benchmark_main, Bencher};
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use std::sync::{mpsc, Arc};
fn spawn_many(b: &mut Bencher) {
const NUM_SPAWN: usize = 10_000;
let rt = rt();
let (tx, rx) = mpsc::sync_channel(1000);
let rem = Arc::new(AtomicUsize::new(0));
b.iter(|| {
rem.store(NUM_SPAWN, Relaxed);
rt.block_on(async {
for _ in 0..NUM_SPAWN {
let tx = tx.clone();
let rem = rem.clone();
tokio::spawn(async move {
if 1 == rem.fetch_sub(1, Relaxed) {
tx.send(()).unwrap();
}
});
}
let _ = rx.recv().unwrap();
});
});
}
fn yield_many(b: &mut Bencher) {
const NUM_YIELD: usize = 1_000;
const TASKS: usize = 200;
let rt = rt();
let (tx, rx) = mpsc::sync_channel(TASKS);
b.iter(move || {
for _ in 0..TASKS {
let tx = tx.clone();
rt.spawn(async move {
for _ in 0..NUM_YIELD {
tokio::task::yield_now().await;
}
tx.send(()).unwrap();
});
}
for _ in 0..TASKS {
let _ = rx.recv().unwrap();
}
});
}
fn ping_pong(b: &mut Bencher) {
const NUM_PINGS: usize = 1_000;
let rt = rt();
let (done_tx, done_rx) = mpsc::sync_channel(1000);
let rem = Arc::new(AtomicUsize::new(0));
b.iter(|| {
let done_tx = done_tx.clone();
let rem = rem.clone();
rem.store(NUM_PINGS, Relaxed);
rt.block_on(async {
tokio::spawn(async move {
for _ in 0..NUM_PINGS {
let rem = rem.clone();
let done_tx = done_tx.clone();
tokio::spawn(async move {
let (tx1, rx1) = oneshot::channel();
let (tx2, rx2) = oneshot::channel();
tokio::spawn(async move {
rx1.await.unwrap();
tx2.send(()).unwrap();
});
tx1.send(()).unwrap();
rx2.await.unwrap();
if 1 == rem.fetch_sub(1, Relaxed) {
done_tx.send(()).unwrap();
}
});
}
});
done_rx.recv().unwrap();
});
});
}
fn chained_spawn(b: &mut Bencher) {
const ITER: usize = 1_000;
let rt = rt();
fn iter(done_tx: mpsc::SyncSender<()>, n: usize) {
if n == 0 {
done_tx.send(()).unwrap();
} else {
tokio::spawn(async move {
iter(done_tx, n - 1);
});
}
}
let (done_tx, done_rx) = mpsc::sync_channel(1000);
b.iter(move || {
let done_tx = done_tx.clone();
rt.block_on(async {
tokio::spawn(async move {
iter(done_tx, ITER);
});
done_rx.recv().unwrap();
});
});
}
fn rt() -> Runtime {
runtime::Builder::new_multi_thread()
.worker_threads(4)
.enable_all()
.build()
.unwrap()
}
benchmark_group!(scheduler, spawn_many, ping_pong, yield_many, chained_spawn,);
benchmark_main!(scheduler);
-95
View File
@@ -1,95 +0,0 @@
//! Benchmark the delay in propagating OS signals to any listeners.
#![cfg(unix)]
use bencher::{benchmark_group, benchmark_main, Bencher};
use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio::runtime;
use tokio::signal::unix::{signal, SignalKind};
use tokio::sync::mpsc;
struct Spinner {
count: usize,
}
impl Future for Spinner {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
if self.count > 3 {
Poll::Ready(())
} else {
self.count += 1;
cx.waker().wake_by_ref();
Poll::Pending
}
}
}
impl Spinner {
fn new() -> Self {
Self { count: 0 }
}
}
pub fn send_signal(signal: libc::c_int) {
use libc::{getpid, kill};
unsafe {
assert_eq!(kill(getpid(), signal), 0);
}
}
fn many_signals(bench: &mut Bencher) {
let num_signals = 10;
let (tx, mut rx) = mpsc::channel(num_signals);
// Intentionally single threaded to measure delays in propagating wakes
let rt = runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let spawn_signal = |kind| {
let tx = tx.clone();
rt.spawn(async move {
let mut signal = signal(kind).expect("failed to create signal");
while signal.recv().await.is_some() {
if tx.send(()).await.is_err() {
break;
}
}
});
};
for _ in 0..num_signals {
// Pick some random signals which don't terminate the test harness
spawn_signal(SignalKind::child());
spawn_signal(SignalKind::io());
}
drop(tx);
// Turn the runtime for a while to ensure that all the spawned
// tasks have been polled at least once
rt.block_on(Spinner::new());
bench.iter(|| {
rt.block_on(async {
send_signal(libc::SIGCHLD);
for _ in 0..num_signals {
rx.recv().await.expect("channel closed");
}
send_signal(libc::SIGIO);
for _ in 0..num_signals {
rx.recv().await.expect("channel closed");
}
});
});
}
benchmark_group!(signal_group, many_signals,);
benchmark_main!(signal_group);
-64
View File
@@ -1,64 +0,0 @@
//! Benchmark spawning a task onto the basic and threaded Tokio executors.
//! This essentially measure the time to enqueue a task in the local and remote
//! case.
use bencher::{black_box, Bencher};
async fn work() -> usize {
let val = 1 + 1;
black_box(val)
}
fn basic_scheduler_local_spawn(bench: &mut Bencher) {
let runtime = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
runtime.block_on(async {
bench.iter(|| {
let h = tokio::spawn(work());
black_box(h);
})
});
}
fn threaded_scheduler_local_spawn(bench: &mut Bencher) {
let runtime = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
runtime.block_on(async {
bench.iter(|| {
let h = tokio::spawn(work());
black_box(h);
})
});
}
fn basic_scheduler_remote_spawn(bench: &mut Bencher) {
let runtime = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
bench.iter(|| {
let h = runtime.spawn(work());
black_box(h);
});
}
fn threaded_scheduler_remote_spawn(bench: &mut Bencher) {
let runtime = tokio::runtime::Builder::new_multi_thread().build().unwrap();
bench.iter(|| {
let h = runtime.spawn(work());
black_box(h);
});
}
bencher::benchmark_group!(
spawn,
basic_scheduler_local_spawn,
threaded_scheduler_local_spawn,
basic_scheduler_remote_spawn,
threaded_scheduler_remote_spawn
);
bencher::benchmark_main!(spawn);
-175
View File
@@ -1,175 +0,0 @@
use bencher::{black_box, Bencher};
use tokio::sync::mpsc;
type Medium = [usize; 64];
type Large = [Medium; 64];
fn rt() -> tokio::runtime::Runtime {
tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap()
}
fn create_1_medium(b: &mut Bencher) {
b.iter(|| {
black_box(&mpsc::channel::<Medium>(1));
});
}
fn create_100_medium(b: &mut Bencher) {
b.iter(|| {
black_box(&mpsc::channel::<Medium>(100));
});
}
fn create_100_000_medium(b: &mut Bencher) {
b.iter(|| {
black_box(&mpsc::channel::<Medium>(100_000));
});
}
fn send_medium(b: &mut Bencher) {
b.iter(|| {
let (tx, mut rx) = mpsc::channel::<Medium>(1000);
let _ = tx.try_send([0; 64]);
rx.try_recv().unwrap();
});
}
fn send_large(b: &mut Bencher) {
b.iter(|| {
let (tx, mut rx) = mpsc::channel::<Large>(1000);
let _ = tx.try_send([[0; 64]; 64]);
rx.try_recv().unwrap();
});
}
fn contention_bounded(b: &mut Bencher) {
let rt = rt();
b.iter(|| {
rt.block_on(async move {
let (tx, mut rx) = mpsc::channel::<usize>(1_000_000);
for _ in 0..5 {
let tx = tx.clone();
tokio::spawn(async move {
for i in 0..1000 {
tx.send(i).await.unwrap();
}
});
}
for _ in 0..1_000 * 5 {
let _ = rx.recv().await;
}
})
});
}
fn contention_bounded_full(b: &mut Bencher) {
let rt = rt();
b.iter(|| {
rt.block_on(async move {
let (tx, mut rx) = mpsc::channel::<usize>(100);
for _ in 0..5 {
let tx = tx.clone();
tokio::spawn(async move {
for i in 0..1000 {
tx.send(i).await.unwrap();
}
});
}
for _ in 0..1_000 * 5 {
let _ = rx.recv().await;
}
})
});
}
fn contention_unbounded(b: &mut Bencher) {
let rt = rt();
b.iter(|| {
rt.block_on(async move {
let (tx, mut rx) = mpsc::unbounded_channel::<usize>();
for _ in 0..5 {
let tx = tx.clone();
tokio::spawn(async move {
for i in 0..1000 {
tx.send(i).unwrap();
}
});
}
for _ in 0..1_000 * 5 {
let _ = rx.recv().await;
}
})
});
}
fn uncontented_bounded(b: &mut Bencher) {
let rt = rt();
b.iter(|| {
rt.block_on(async move {
let (tx, mut rx) = mpsc::channel::<usize>(1_000_000);
for i in 0..5000 {
tx.send(i).await.unwrap();
}
for _ in 0..5_000 {
let _ = rx.recv().await;
}
})
});
}
fn uncontented_unbounded(b: &mut Bencher) {
let rt = rt();
b.iter(|| {
rt.block_on(async move {
let (tx, mut rx) = mpsc::unbounded_channel::<usize>();
for i in 0..5000 {
tx.send(i).unwrap();
}
for _ in 0..5_000 {
let _ = rx.recv().await;
}
})
});
}
bencher::benchmark_group!(
create,
create_1_medium,
create_100_medium,
create_100_000_medium
);
bencher::benchmark_group!(send, send_medium, send_large);
bencher::benchmark_group!(
contention,
contention_bounded,
contention_bounded_full,
contention_unbounded,
uncontented_bounded,
uncontented_unbounded
);
bencher::benchmark_main!(create, send, contention);
-142
View File
@@ -1,142 +0,0 @@
use bencher::{black_box, Bencher};
use std::sync::Arc;
use tokio::{sync::RwLock, task};
fn read_uncontended(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap();
let lock = Arc::new(RwLock::new(()));
b.iter(|| {
let lock = lock.clone();
rt.block_on(async move {
for _ in 0..6 {
let read = lock.read().await;
black_box(read);
}
})
});
}
fn read_concurrent_uncontended_multi(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap();
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
}
let lock = Arc::new(RwLock::new(()));
b.iter(|| {
let lock = lock.clone();
rt.block_on(async move {
let j = tokio::try_join! {
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone()))
};
j.unwrap();
})
});
}
fn read_concurrent_uncontended(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
}
let lock = Arc::new(RwLock::new(()));
b.iter(|| {
let lock = lock.clone();
rt.block_on(async move {
tokio::join! {
task(lock.clone()),
task(lock.clone()),
task(lock.clone()),
task(lock.clone()),
task(lock.clone()),
task(lock.clone())
};
})
});
}
fn read_concurrent_contended_multi(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap();
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
}
let lock = Arc::new(RwLock::new(()));
b.iter(|| {
let lock = lock.clone();
rt.block_on(async move {
let write = lock.write().await;
let j = tokio::try_join! {
async move { drop(write); Ok(()) },
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
task::spawn(task(lock.clone())),
};
j.unwrap();
})
});
}
fn read_concurrent_contended(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
}
let lock = Arc::new(RwLock::new(()));
b.iter(|| {
let lock = lock.clone();
rt.block_on(async move {
let write = lock.write().await;
tokio::join! {
async move { drop(write) },
task(lock.clone()),
task(lock.clone()),
task(lock.clone()),
task(lock.clone()),
task(lock.clone()),
};
})
});
}
bencher::benchmark_group!(
sync_rwlock,
read_uncontended,
read_concurrent_uncontended,
read_concurrent_uncontended_multi,
read_concurrent_contended,
read_concurrent_contended_multi
);
bencher::benchmark_main!(sync_rwlock);
-125
View File
@@ -1,125 +0,0 @@
use bencher::Bencher;
use std::sync::Arc;
use tokio::{sync::Semaphore, task};
fn uncontended(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap();
let s = Arc::new(Semaphore::new(10));
b.iter(|| {
let s = s.clone();
rt.block_on(async move {
for _ in 0..6 {
let permit = s.acquire().await;
drop(permit);
}
})
});
}
async fn task(s: Arc<Semaphore>) {
let permit = s.acquire().await;
drop(permit);
}
fn uncontended_concurrent_multi(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap();
let s = Arc::new(Semaphore::new(10));
b.iter(|| {
let s = s.clone();
rt.block_on(async move {
let j = tokio::try_join! {
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone()))
};
j.unwrap();
})
});
}
fn uncontended_concurrent_single(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
let s = Arc::new(Semaphore::new(10));
b.iter(|| {
let s = s.clone();
rt.block_on(async move {
tokio::join! {
task(s.clone()),
task(s.clone()),
task(s.clone()),
task(s.clone()),
task(s.clone()),
task(s.clone())
};
})
});
}
fn contended_concurrent_multi(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(6)
.build()
.unwrap();
let s = Arc::new(Semaphore::new(5));
b.iter(|| {
let s = s.clone();
rt.block_on(async move {
let j = tokio::try_join! {
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone())),
task::spawn(task(s.clone()))
};
j.unwrap();
})
});
}
fn contended_concurrent_single(b: &mut Bencher) {
let rt = tokio::runtime::Builder::new_current_thread()
.build()
.unwrap();
let s = Arc::new(Semaphore::new(5));
b.iter(|| {
let s = s.clone();
rt.block_on(async move {
tokio::join! {
task(s.clone()),
task(s.clone()),
task(s.clone()),
task(s.clone()),
task(s.clone()),
task(s.clone())
};
})
});
}
bencher::benchmark_group!(
sync_semaphore,
uncontended,
uncontended_concurrent_multi,
uncontended_concurrent_single,
contended_concurrent_multi,
contended_concurrent_single
);
bencher::benchmark_main!(sync_semaphore);
-121
View File
@@ -1,121 +0,0 @@
#!/usr/bin/env bash
set -e
USAGE="Publish a new release of a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-v, --verbose Use verbose Cargo output
-d, --dry-run Perform a dry run (do not publish or tag the release)
-h, --help Show this help text and exit"
DRY_RUN=""
VERBOSE=""
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
verify() {
status "Verifying" "if $CRATE v$VERSION can be released"
ACTUAL=$(cargo pkgid | sed -n 's/.*#\(.*\)/\1/p')
if [ "$ACTUAL" != "$VERSION" ]; then
err "expected to release version $VERSION, but Cargo.toml contained $ACTUAL"
exit 1
fi
if git tag -l | grep -Fxq "$TAG" ; then
err "git tag \`$TAG\` already exists"
exit 1
fi
PATH_DEPS=$(grep -F "path = \"" Cargo.toml | sed -e 's/^/ /')
if [ -n "$PATH_DEPS" ]; then
err "crate \`$CRATE\` contained path dependencies:\n$PATH_DEPS"
echo "path dependencies must be removed prior to release"
exit 1
fi
}
release() {
status "Releasing" "$CRATE v$VERSION"
cargo package $VERBOSE
cargo publish $VERBOSE $DRY_RUN
status "Tagging" "$TAG"
if [ -n "$DRY_RUN" ]; then
echo "# git tag $TAG && git push --tags"
else
git tag "$TAG" && git push --tags
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-v|--verbose)
VERBOSE="--verbose"
set +x
shift
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
(cd "$CRATE" && verify && release )
else
err "no such crate \"$CRATE\""
exit 1
fi
-118
View File
@@ -1,118 +0,0 @@
#!/usr/bin/env bash
set -e
USAGE="Update links to docs.rs in a tokio crate
USAGE:
$(basename "$0") [OPTIONS] [CRATE] [VERSION]
OPTIONS:
-d, --dry-run Perform a dry run (do not modify any file)
-h, --help Show this help text and exit"
err() {
echo -e "\e[31m\e[1merror:\e[0m $@" 1>&2;
}
status() {
WIDTH=12
printf "\e[32m\e[1m%${WIDTH}s\e[0m %s\n" "$1" "$2"
}
c1grep() { grep "$@" || test $? = 1; }
update_versions_in_doc() {
# Print what is being/would be done
if [ -n "$DRY_RUN" ]; then
local MSG="Would change:"
else
local MSG="Updating:"
fi
git grep -lr "docs.rs/$CRATE/" \
| xargs sed --quiet \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|gp" \
| sed -e "s/^/$MSG /"
# Apply changes if not in dry run
if [ -z "$DRY_RUN" ]; then
git grep -lr "docs.rs/$CRATE/" \
| xargs sed -i \
-E "s|docs.rs/$CRATE/[0-9.]+|docs.rs/$CRATE/$VERSION|g"
fi
}
update() {
update_versions_in_doc
}
show_outdated() {
OUTDATED=$(git grep -rn "docs.rs/$CRATE/" \
| c1grep -v "$VERSION" \
| sed -e 's/^/ - /')
if [[ -n "$OUTDATED" ]]; then
echo "Found the following links to docs.rs with an outdated version:"
echo "$OUTDATED"
echo
else
echo "Nothing to do."
exit 1
fi
}
while [[ $# -gt 0 ]]
do
case "$1" in
-h|--help)
echo "$USAGE"
exit 0
;;
-d|--dry-run)
DRY_RUN="--dry-run"
shift
;;
-*)
err "unknown flag \"$1\""
echo "$USAGE"
exit 1
;;
*) # crate or version
if [ -z "$CRATE" ]; then
CRATE="$1"
elif [ -z "$VERSION" ]; then
VERSION="$1"
else
err "unknown positional argument \"$1\""
echo "$USAGE"
exit 1
fi
shift
;;
esac
done
# set -- "${POSITIONAL[@]}"
if [ -z "$VERSION" ]; then
err "no version specified!"
HELP=1
fi
if [ -n "$CRATE" ]; then
TAG="$CRATE-$VERSION"
else
err "no crate specified!"
HELP=1
fi
if [ -n "$HELP" ]; then
echo "$USAGE"
exit 1
fi
if [ -d "$CRATE" ]; then
# Does not cd in order to update everywhere
show_outdated && update
else
err "no such crate \"$CRATE\""
exit 1
fi
-65
View File
@@ -1,65 +0,0 @@
[package]
name = "examples"
version = "0.0.0"
publish = false
edition = "2018"
# If you copy one of the examples into a new project, you should be using
# [dependencies] instead.
[dev-dependencies]
tokio = { version = "0.3.0", path = "../tokio", features = ["full", "tracing"] }
tracing = "0.1"
tracing-subscriber = { version = "0.2.7", default-features = false, features = ["fmt", "ansi", "env-filter", "chrono", "tracing-log"] }
tokio-util = { version = "0.5.0", path = "../tokio-util", features = ["full"] }
bytes = "0.6"
futures = "0.3.0"
http = "0.2"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
httparse = "1.0"
time = "0.1"
[[example]]
name = "chat"
path = "chat.rs"
[[example]]
name = "connect"
path = "connect.rs"
[[example]]
name = "echo-udp"
path = "echo-udp.rs"
[[example]]
name = "echo"
path = "echo.rs"
[[example]]
name = "hello_world"
path = "hello_world.rs"
[[example]]
name = "print_each_packet"
path = "print_each_packet.rs"
[[example]]
name = "proxy"
path = "proxy.rs"
[[example]]
name = "tinydb"
path = "tinydb.rs"
[[example]]
name = "udp-client"
path = "udp-client.rs"
[[example]]
name = "udp-codec"
path = "udp-codec.rs"
[[example]]
name = "tinyhttp"
path = "tinyhttp.rs"
-23
View File
@@ -1,23 +0,0 @@
## Examples of how to use Tokio
This directory contains a number of examples showcasing various capabilities of
the `tokio` crate.
All examples can be executed with:
```
cargo run --example $name
```
A good starting point for the examples would be [`hello_world`](hello_world.rs)
and [`echo`](echo.rs). Additionally [the tokio website][tokioweb] contains
additional guides for some of the examples.
For a larger "real world" example, see the [`mini-redis`][redis] repository.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
[tokioweb]: https://tokio.rs/tokio/tutorial
[redis]: https://github.com/tokio-rs/mini-redis
-278
View File
@@ -1,278 +0,0 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This example is explicitly more verbose than it has to be. This is to
//! illustrate more concepts.
//!
//! A chat server for telnet clients. After a telnet client connects, the first
//! line should contain the client's name. After that, all lines sent by a
//! client are broadcasted to all other connected clients.
//!
//! Because the client is telnet, lines are delimited by "\r\n".
//!
//! You can test this out by running:
//!
//! cargo run --example chat
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
//! You can run the `telnet` command in any number of additional windows.
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![warn(rust_2018_idioms)]
use tokio::net::{TcpListener, TcpStream};
use tokio::stream::{Stream, StreamExt};
use tokio::sync::{mpsc, Mutex};
use tokio_util::codec::{Framed, LinesCodec, LinesCodecError};
use futures::SinkExt;
use std::collections::HashMap;
use std::env;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use std::pin::Pin;
use std::sync::Arc;
use std::task::{Context, Poll};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
use tracing_subscriber::{fmt::format::FmtSpan, EnvFilter};
// Configure a `tracing` subscriber that logs traces emitted by the chat
// server.
tracing_subscriber::fmt()
// Filter what traces are displayed based on the RUST_LOG environment
// variable.
//
// Traces emitted by the example code will always be displayed. You
// can set `RUST_LOG=tokio=trace` to enable additional traces emitted by
// Tokio itself.
.with_env_filter(EnvFilter::from_default_env().add_directive("chat=info".parse()?))
// Log events when `tracing` spans are created, entered, exited, or
// closed. When Tokio's internal tracing support is enabled (as
// described above), this can be used to track the lifecycle of spawned
// tasks on the Tokio runtime.
.with_span_events(FmtSpan::FULL)
// Set this subscriber as the default, to collect all traces emitted by
// the program.
.init();
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:6142".to_string());
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).await?;
tracing::info!("server running on {}", addr);
loop {
// Asynchronously wait for an inbound TcpStream.
let (stream, addr) = listener.accept().await?;
// Clone a handle to the `Shared` state for the new connection.
let state = Arc::clone(&state);
// Spawn our handler to be run asynchronously.
tokio::spawn(async move {
tracing::debug!("accepted connection");
if let Err(e) = process(state, stream, addr).await {
tracing::info!("an error occurred; error = {:?}", e);
}
});
}
}
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<String>;
/// Shorthand for the receive half of the message channel.
type Rx = mpsc::UnboundedReceiver<String>;
/// Data that is shared between all peers in the chat server.
///
/// This is the set of `Tx` handles for all connected clients. Whenever a
/// message is received from a client, it is broadcasted to all peers by
/// iterating over the `peers` entries and sending a copy of the message on each
/// `Tx`.
struct Shared {
peers: HashMap<SocketAddr, Tx>,
}
/// The state for each connected client.
struct Peer {
/// The TCP socket wrapped with the `Lines` codec, defined below.
///
/// This handles sending and receiving data on the socket. When using
/// `Lines`, we can work at the line level instead of having to manage the
/// raw byte operations.
lines: Framed<TcpStream, LinesCodec>,
/// Receive half of the message channel.
///
/// This is used to receive messages from peers. When a message is received
/// off of this `Rx`, it will be written to the socket.
rx: Rx,
}
impl Shared {
/// Create a new, empty, instance of `Shared`.
fn new() -> Self {
Shared {
peers: HashMap::new(),
}
}
/// Send a `LineCodec` encoded message to every peer, except
/// for the sender.
async fn broadcast(&mut self, sender: SocketAddr, message: &str) {
for peer in self.peers.iter_mut() {
if *peer.0 != sender {
let _ = peer.1.send(message.into());
}
}
}
}
impl Peer {
/// Create a new instance of `Peer`.
async fn new(
state: Arc<Mutex<Shared>>,
lines: Framed<TcpStream, LinesCodec>,
) -> io::Result<Peer> {
// Get the client socket address
let addr = lines.get_ref().peer_addr()?;
// Create a channel for this peer
let (tx, rx) = mpsc::unbounded_channel();
// Add an entry for this `Peer` in the shared state map.
state.lock().await.peers.insert(addr, tx);
Ok(Peer { lines, rx })
}
}
#[derive(Debug)]
enum Message {
/// A message that should be broadcasted to others.
Broadcast(String),
/// A message that should be received by a client
Received(String),
}
// Peer implements `Stream` in a way that polls both the `Rx`, and `Framed` types.
// A message is produced whenever an event is ready until the `Framed` stream returns `None`.
impl Stream for Peer {
type Item = Result<Message, LinesCodecError>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
// First poll the `UnboundedReceiver`.
if let Poll::Ready(Some(v)) = Pin::new(&mut self.rx).poll_next(cx) {
return Poll::Ready(Some(Ok(Message::Received(v))));
}
// Secondly poll the `Framed` stream.
let result: Option<_> = futures::ready!(Pin::new(&mut self.lines).poll_next(cx));
Poll::Ready(match result {
// We've received a message we should broadcast to others.
Some(Ok(message)) => Some(Ok(Message::Broadcast(message))),
// An error occurred.
Some(Err(e)) => Some(Err(e)),
// The stream has been exhausted.
None => None,
})
}
}
/// Process an individual chat client
async fn process(
state: Arc<Mutex<Shared>>,
stream: TcpStream,
addr: SocketAddr,
) -> Result<(), Box<dyn Error>> {
let mut lines = Framed::new(stream, LinesCodec::new());
// Send a prompt to the client to enter their username.
lines.send("Please enter your username:").await?;
// Read the first line from the `LineCodec` stream to get the username.
let username = match lines.next().await {
Some(Ok(line)) => line,
// We didn't get a line so we return early here.
_ => {
tracing::error!("Failed to get username from {}. Client disconnected.", addr);
return Ok(());
}
};
// Register our peer with state which internally sets up some channels.
let mut peer = Peer::new(state.clone(), lines).await?;
// A client has connected, let's let everyone know.
{
let mut state = state.lock().await;
let msg = format!("{} has joined the chat", username);
tracing::info!("{}", msg);
state.broadcast(addr, &msg).await;
}
// Process incoming messages until our stream is exhausted by a disconnect.
while let Some(result) = peer.next().await {
match result {
// A message was received from the current user, we should
// broadcast this message to the other users.
Ok(Message::Broadcast(msg)) => {
let mut state = state.lock().await;
let msg = format!("{}: {}", username, msg);
state.broadcast(addr, &msg).await;
}
// A message was received from a peer. Send it to the
// current user.
Ok(Message::Received(msg)) => {
peer.lines.send(&msg).await?;
}
Err(e) => {
tracing::error!(
"an error occurred while processing messages for {}; error = {:?}",
username,
e
);
}
}
}
// If this section is reached it means that the client was disconnected!
// Let's let everyone still connected know about it.
{
let mut state = state.lock().await;
state.peers.remove(&addr);
let msg = format!("{} has left the chat", username);
tracing::info!("{}", msg);
state.broadcast(addr, &msg).await;
}
Ok(())
}
-147
View File
@@ -1,147 +0,0 @@
//! An example of hooking up stdin/stdout to either a TCP or UDP stream.
//!
//! This example will connect to a socket address specified in the argument list
//! and then forward all data read on stdin to the server, printing out all data
//! received on stdout. An optional `--udp` argument can be passed to specify
//! that the connection should be made over UDP instead of TCP, translating each
//! line entered on stdin to a UDP packet to be sent to the remote address.
//!
//! Note that this is not currently optimized for performance, especially
//! around buffer management. Rather it's intended to show an example of
//! working with a client.
//!
//! This example can be quite useful when interacting with the other examples in
//! this repository! Many of them recommend running this as a simple "hook up
//! stdin/stdout to a server" to get up and running.
#![warn(rust_2018_idioms)]
use futures::StreamExt;
use tokio::io;
use tokio_util::codec::{BytesCodec, FramedRead, FramedWrite};
use std::env;
use std::error::Error;
use std::net::SocketAddr;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
Some(i) => {
args.remove(i);
false
}
None => true,
};
// Parse what address we're going to connect to
let addr = args
.first()
.ok_or("this program requires at least one argument")?;
let addr = addr.parse::<SocketAddr>()?;
let stdin = FramedRead::new(io::stdin(), BytesCodec::new());
let stdin = stdin.map(|i| i.map(|bytes| bytes.freeze()));
let stdout = FramedWrite::new(io::stdout(), BytesCodec::new());
if tcp {
tcp::connect(&addr, stdin, stdout).await?;
} else {
udp::connect(&addr, stdin, stdout).await?;
}
Ok(())
}
mod tcp {
use bytes::Bytes;
use futures::{future, Sink, SinkExt, Stream, StreamExt};
use std::{error::Error, io, net::SocketAddr};
use tokio::net::TcpStream;
use tokio_util::codec::{BytesCodec, FramedRead, FramedWrite};
pub async fn connect(
addr: &SocketAddr,
mut stdin: impl Stream<Item = Result<Bytes, io::Error>> + Unpin,
mut stdout: impl Sink<Bytes, Error = io::Error> + Unpin,
) -> Result<(), Box<dyn Error>> {
let mut stream = TcpStream::connect(addr).await?;
let (r, w) = stream.split();
let mut sink = FramedWrite::new(w, BytesCodec::new());
// filter map Result<BytesMut, Error> stream into just a Bytes stream to match stdout Sink
// on the event of an Error, log the error and end the stream
let mut stream = FramedRead::new(r, BytesCodec::new())
.filter_map(|i| match i {
//BytesMut into Bytes
Ok(i) => future::ready(Some(i.freeze())),
Err(e) => {
println!("failed to read from socket; error={}", e);
future::ready(None)
}
})
.map(Ok);
match future::join(sink.send_all(&mut stdin), stdout.send_all(&mut stream)).await {
(Err(e), _) | (_, Err(e)) => Err(e.into()),
_ => Ok(()),
}
}
}
mod udp {
use bytes::Bytes;
use futures::{future, Sink, SinkExt, Stream, StreamExt};
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use tokio::net::UdpSocket;
pub async fn connect(
addr: &SocketAddr,
stdin: impl Stream<Item = Result<Bytes, io::Error>> + Unpin,
stdout: impl Sink<Bytes, Error = io::Error> + Unpin,
) -> Result<(), Box<dyn Error>> {
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let bind_addr = if addr.ip().is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
};
let socket = UdpSocket::bind(&bind_addr).await?;
socket.connect(addr).await?;
future::try_join(send(stdin, &socket), recv(stdout, &socket)).await?;
Ok(())
}
async fn send(
mut stdin: impl Stream<Item = Result<Bytes, io::Error>> + Unpin,
writer: &UdpSocket,
) -> Result<(), io::Error> {
while let Some(item) = stdin.next().await {
let buf = item?;
writer.send(&buf[..]).await?;
}
Ok(())
}
async fn recv(
mut stdout: impl Sink<Bytes, Error = io::Error> + Unpin,
reader: &UdpSocket,
) -> Result<(), io::Error> {
loop {
let mut buf = vec![0; 1024];
let n = reader.recv(&mut buf[..]).await?;
if n > 0 {
stdout.send(Bytes::from(buf)).await?;
}
}
}
}
-70
View File
@@ -1,70 +0,0 @@
//! An UDP echo server that just sends back everything that it receives.
//!
//! If you're on Unix you can test this out by in one terminal executing:
//!
//! cargo run --example echo-udp
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect -- --udp 127.0.0.1:8080
//!
//! Each line you type in to the `nc` terminal should be echo'd back to you!
#![warn(rust_2018_idioms)]
use std::error::Error;
use std::net::SocketAddr;
use std::{env, io};
use tokio::net::UdpSocket;
struct Server {
socket: UdpSocket,
buf: Vec<u8>,
to_send: Option<(usize, SocketAddr)>,
}
impl Server {
async fn run(self) -> Result<(), io::Error> {
let Server {
socket,
mut buf,
mut to_send,
} = self;
loop {
// First we check to see if there's a message we need to echo back.
// If so then we try to send it back to the original source, waiting
// until it's writable and we're able to do so.
if let Some((size, peer)) = to_send {
let amt = socket.send_to(&buf[..size], &peer).await?;
println!("Echoed {}/{} bytes to {}", amt, size, peer);
}
// If we're here then `to_send` is `None`, so we take a look for the
// next message we're going to echo back.
to_send = Some(socket.recv_from(&mut buf).await?);
}
}
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
let socket = UdpSocket::bind(&addr).await?;
println!("Listening on: {}", socket.local_addr()?);
let server = Server {
socket,
buf: vec![0; 1024],
to_send: None,
};
// This starts the server task.
server.run().await?;
Ok(())
}
-78
View File
@@ -1,78 +0,0 @@
//! A "hello world" echo server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! write back everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example echo
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be echo'd back to
//! you! If you open up multiple terminals running the `connect` example you
//! should be able to see them all make progress simultaneously.
#![warn(rust_2018_idioms)]
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpListener;
use std::env;
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop.
let listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
loop {
// Asynchronously wait for an inbound socket.
let (mut socket, _) = listener.accept().await?;
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(async move {
let mut buf = [0; 1024];
// In a loop, read data from the socket and write the data back.
loop {
let n = socket
.read(&mut buf)
.await
.expect("failed to read data from socket");
if n == 0 {
return;
}
socket
.write_all(&buf[0..n])
.await
.expect("failed to write data to socket");
}
});
}
}
-33
View File
@@ -1,33 +0,0 @@
//! Hello world server.
//!
//! A simple client that opens a TCP stream, writes "hello world\n", and closes
//! the connection.
//!
//! You can test this out by running:
//!
//! ncat -l 6142
//!
//! And then in another terminal run:
//!
//! cargo run --example hello_world
#![warn(rust_2018_idioms)]
use tokio::io::AsyncWriteExt;
use tokio::net::TcpStream;
use std::error::Error;
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn Error>> {
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpStream, which is fully async.
let mut stream = TcpStream::connect("127.0.0.1:6142").await?;
println!("created stream");
let result = stream.write(b"hello world\n").await;
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
}
+17
View File
@@ -0,0 +1,17 @@
extern crate futures;
extern crate tokio_core;
extern crate tokio_signal;
use futures::stream::Stream;
use tokio_core::reactor::Core;
fn main() {
let mut core = Core::new().unwrap();
let ctrlc = tokio_signal::ctrl_c(&core.handle());
let stream = core.run(ctrlc).unwrap();
core.run(stream.for_each(|()| {
println!("Ctrl-C received!");
Ok(())
})).unwrap();
}
-106
View File
@@ -1,106 +0,0 @@
//! A "print-each-packet" server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! put down in the stdout everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example print\_each\_packet
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be written to terminal!
//!
//! Minimal js example:
//!
//! ```js
//! var net = require("net");
//!
//! var listenPort = 8080;
//!
//! var server = net.createServer(function (socket) {
//! socket.on("data", function (bytes) {
//! console.log("bytes", bytes);
//! });
//!
//! socket.on("end", function() {
//! console.log("Socket received FIN packet and closed connection");
//! });
//! socket.on("error", function (error) {
//! console.log("Socket closed with error", error);
//! });
//!
//! socket.on("close", function (with_error) {
//! if (with_error) {
//! console.log("Socket closed with result: Err(SomeError)");
//! } else {
//! console.log("Socket closed with result: Ok(())");
//! }
//! });
//!
//! });
//!
//! server.listen(listenPort);
//!
//! console.log("Listening on:", listenPort);
//! ```
//!
#![warn(rust_2018_idioms)]
use tokio::net::TcpListener;
use tokio::stream::StreamExt;
use tokio_util::codec::{BytesCodec, Decoder};
use std::env;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
loop {
// Asynchronously wait for an inbound socket.
let (socket, _) = listener.accept().await?;
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(async move {
// We're parsing each socket with the `BytesCodec` included in `tokio::codec`.
let mut framed = BytesCodec::new().framed(socket);
// We loop while there are messages coming from the Stream `framed`.
// The stream will return None once the client disconnects.
while let Some(message) = framed.next().await {
match message {
Ok(bytes) => println!("bytes: {:?}", bytes),
Err(err) => println!("Socket closed with error: {:?}", err),
}
}
println!("Socket received FIN packet and closed connection");
});
}
}
-80
View File
@@ -1,80 +0,0 @@
//! A proxy that forwards data to another server and forwards that server's
//! responses back to clients.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! You can showcase this by running this in one terminal:
//!
//! cargo run --example proxy
//!
//! This in another terminal
//!
//! cargo run --example echo
//!
//! And finally this in another terminal
//!
//! cargo run --example connect 127.0.0.1:8081
//!
//! This final terminal will connect to our proxy, which will in turn connect to
//! the echo server, and you'll be able to see data flowing between them.
#![warn(rust_2018_idioms)]
use tokio::io;
use tokio::io::AsyncWriteExt;
use tokio::net::{TcpListener, TcpStream};
use futures::future::try_join;
use futures::FutureExt;
use std::env;
use std::error::Error;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let listen_addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8081".to_string());
let server_addr = env::args()
.nth(2)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let listener = TcpListener::bind(listen_addr).await?;
while let Ok((inbound, _)) = listener.accept().await {
let transfer = transfer(inbound, server_addr.clone()).map(|r| {
if let Err(e) = r {
println!("Failed to transfer; error={}", e);
}
});
tokio::spawn(transfer);
}
Ok(())
}
async fn transfer(mut inbound: TcpStream, proxy_addr: String) -> Result<(), Box<dyn Error>> {
let mut outbound = TcpStream::connect(proxy_addr).await?;
let (mut ri, mut wi) = inbound.split();
let (mut ro, mut wo) = outbound.split();
let client_to_server = async {
io::copy(&mut ri, &mut wo).await?;
wo.shutdown().await
};
let server_to_client = async {
io::copy(&mut ro, &mut wi).await?;
wi.shutdown().await
};
try_join(client_to_server, server_to_client).await?;
Ok(())
}
-224
View File
@@ -1,224 +0,0 @@
//! A "tiny database" and accompanying protocol
//!
//! This example shows the usage of shared state amongst all connected clients,
//! namely a database of key/value pairs. Each connected client can send a
//! series of GET/SET commands to query the current value of a key or set the
//! value of a key.
//!
//! This example has a simple protocol you can use to interact with the server.
//! To run, first run this in one terminal window:
//!
//! cargo run --example tinydb
//!
//! and next in another windows run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! In the `connect` window you can type in commands where when you hit enter
//! you'll get a response from the server for that command. An example session
//! is:
//!
//!
//! $ cargo run --example connect 127.0.0.1:8080
//! GET foo
//! foo = bar
//! GET FOOBAR
//! error: no key FOOBAR
//! SET FOOBAR my awesome string
//! set FOOBAR = `my awesome string`, previous: None
//! SET foo tokio
//! set foo = `tokio`, previous: Some("bar")
//! GET foo
//! foo = tokio
//!
//! Namely you can issue two forms of commands:
//!
//! * `GET $key` - this will fetch the value of `$key` from the database and
//! return it. The server's database is initially populated with the key `foo`
//! set to the value `bar`
//! * `SET $key $value` - this will set the value of `$key` to `$value`,
//! returning the previous value, if any.
#![warn(rust_2018_idioms)]
use tokio::net::TcpListener;
use tokio::stream::StreamExt;
use tokio_util::codec::{Framed, LinesCodec};
use futures::SinkExt;
use std::collections::HashMap;
use std::env;
use std::error::Error;
use std::sync::{Arc, Mutex};
/// The in-memory database shared amongst all clients.
///
/// This database will be shared via `Arc`, so to mutate the internal map we're
/// going to use a `Mutex` for interior mutability.
struct Database {
map: Mutex<HashMap<String, String>>,
}
/// Possible requests our clients can send us
enum Request {
Get { key: String },
Set { key: String, value: String },
}
/// Responses to the `Request` commands above
enum Response {
Value {
key: String,
value: String,
},
Set {
key: String,
value: String,
previous: Option<String>,
},
Error {
msg: String,
},
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
let listener = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
// clients. We populate the initial database and then create the `Database`
// structure. Note the usage of `Arc` here which will be used to ensure that
// each independently spawned client will have a reference to the in-memory
// database.
let mut initial_db = HashMap::new();
initial_db.insert("foo".to_string(), "bar".to_string());
let db = Arc::new(Database {
map: Mutex::new(initial_db),
});
loop {
match listener.accept().await {
Ok((socket, _)) => {
// After getting a new connection first we see a clone of the database
// being created, which is creating a new reference for this connected
// client to use.
let db = db.clone();
// Like with other small servers, we'll `spawn` this client to ensure it
// runs concurrently with all other clients. The `move` keyword is used
// here to move ownership of our db handle into the async closure.
tokio::spawn(async move {
// Since our protocol is line-based we use `tokio_codecs`'s `LineCodec`
// to convert our stream of bytes, `socket`, into a `Stream` of lines
// as well as convert our line based responses into a stream of bytes.
let mut lines = Framed::new(socket, LinesCodec::new());
// Here for every line we get back from the `Framed` decoder,
// we parse the request, and if it's valid we generate a response
// based on the values in the database.
while let Some(result) = lines.next().await {
match result {
Ok(line) => {
let response = handle_request(&line, &db);
let response = response.serialize();
if let Err(e) = lines.send(response.as_str()).await {
println!("error on sending response; error = {:?}", e);
}
}
Err(e) => {
println!("error on decoding from socket; error = {:?}", e);
}
}
}
// The connection will be closed at this point as `lines.next()` has returned `None`.
});
}
Err(e) => println!("error accepting socket; error = {:?}", e),
}
}
}
fn handle_request(line: &str, db: &Arc<Database>) -> Response {
let request = match Request::parse(&line) {
Ok(req) => req,
Err(e) => return Response::Error { msg: e },
};
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => match db.get(&key) {
Some(value) => Response::Value {
key,
value: value.clone(),
},
None => Response::Error {
msg: format!("no key {}", key),
},
},
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set {
key,
value,
previous,
}
}
}
}
impl Request {
fn parse(input: &str) -> Result<Request, String> {
let mut parts = input.splitn(3, ' ');
match parts.next() {
Some("GET") => {
let key = parts.next().ok_or("GET must be followed by a key")?;
if parts.next().is_some() {
return Err("GET's key must not be followed by anything".into());
}
Ok(Request::Get {
key: key.to_string(),
})
}
Some("SET") => {
let key = match parts.next() {
Some(key) => key,
None => return Err("SET must be followed by a key".into()),
};
let value = match parts.next() {
Some(value) => value,
None => return Err("SET needs a value".into()),
};
Ok(Request::Set {
key: key.to_string(),
value: value.to_string(),
})
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err("empty input".into()),
}
}
}
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => format!("{} = {}", key, value),
Response::Set {
ref key,
ref value,
ref previous,
} => format!("set {} = `{}`, previous: {:?}", key, value, previous),
Response::Error { ref msg } => format!("error: {}", msg),
}
}
}
-301
View File
@@ -1,301 +0,0 @@
//! A "tiny" example of HTTP request/response handling using transports.
//!
//! This example is intended for *learning purposes* to see how various pieces
//! hook up together and how HTTP can get up and running. Note that this example
//! is written with the restriction that it *can't* use any "big" library other
//! than Tokio, if you'd like a "real world" HTTP library you likely want a
//! crate like Hyper.
//!
//! Code here is based on the `echo-threads` example and implements two paths,
//! the `/plaintext` and `/json` routes to respond with some text and json,
//! respectively. By default this will run I/O on all the cores your system has
//! available, and it doesn't support HTTP request bodies.
#![warn(rust_2018_idioms)]
use bytes::BytesMut;
use futures::SinkExt;
use http::{header::HeaderValue, Request, Response, StatusCode};
#[macro_use]
extern crate serde_derive;
use std::{env, error::Error, fmt, io};
use tokio::net::{TcpListener, TcpStream};
use tokio::stream::StreamExt;
use tokio_util::codec::{Decoder, Encoder, Framed};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Parse the arguments, bind the TCP socket we'll be listening to, spin up
// our worker threads, and start shipping sockets to those worker threads.
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".to_string());
let server = TcpListener::bind(&addr).await?;
println!("Listening on: {}", addr);
loop {
let (stream, _) = server.accept().await?;
tokio::spawn(async move {
if let Err(e) = process(stream).await {
println!("failed to process connection; error = {}", e);
}
});
}
}
async fn process(stream: TcpStream) -> Result<(), Box<dyn Error>> {
let mut transport = Framed::new(stream, Http);
while let Some(request) = transport.next().await {
match request {
Ok(request) => {
let response = respond(request).await?;
transport.send(response).await?;
}
Err(e) => return Err(e.into()),
}
}
Ok(())
}
async fn respond(req: Request<()>) -> Result<Response<String>, Box<dyn Error>> {
let mut response = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
response = response.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
"/json" => {
response = response.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
}
serde_json::to_string(&Message {
message: "Hello, World!",
})?
}
_ => {
response = response.status(StatusCode::NOT_FOUND);
String::new()
}
};
let response = response
.body(body)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Ok(response)
}
struct Http;
/// Implementation of encoding an HTTP response into a `BytesMut`, basically
/// just writing out an HTTP/1.1 response.
impl Encoder<Response<String>> for Http {
type Error = io::Error;
fn encode(&mut self, item: Response<String>, dst: &mut BytesMut) -> io::Result<()> {
use std::fmt::Write;
write!(
BytesWrite(dst),
"\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
",
item.status(),
item.body().len(),
date::now()
)
.unwrap();
for (k, v) in item.headers() {
dst.extend_from_slice(k.as_str().as_bytes());
dst.extend_from_slice(b": ");
dst.extend_from_slice(v.as_bytes());
dst.extend_from_slice(b"\r\n");
}
dst.extend_from_slice(b"\r\n");
dst.extend_from_slice(item.body().as_bytes());
return Ok(());
// Right now `write!` on `Vec<u8>` goes through io::Write and is not
// super speedy, so inline a less-crufty implementation here which
// doesn't go through io::Error.
struct BytesWrite<'a>(&'a mut BytesMut);
impl fmt::Write for BytesWrite<'_> {
fn write_str(&mut self, s: &str) -> fmt::Result {
self.0.extend_from_slice(s.as_bytes());
Ok(())
}
fn write_fmt(&mut self, args: fmt::Arguments<'_>) -> fmt::Result {
fmt::write(self, args)
}
}
}
}
/// Implementation of decoding an HTTP request from the bytes we've read so far.
/// This leverages the `httparse` crate to do the actual parsing and then we use
/// that information to construct an instance of a `http::Request` object,
/// trying to avoid allocations where possible.
impl Decoder for Http {
type Item = Request<()>;
type Error = io::Error;
fn decode(&mut self, src: &mut BytesMut) -> io::Result<Option<Request<()>>> {
// TODO: we should grow this headers array if parsing fails and asks
// for more headers
let mut headers = [None; 16];
let (method, path, version, amt) = {
let mut parsed_headers = [httparse::EMPTY_HEADER; 16];
let mut r = httparse::Request::new(&mut parsed_headers);
let status = r.parse(src).map_err(|e| {
let msg = format!("failed to parse http request: {:?}", e);
io::Error::new(io::ErrorKind::Other, msg)
})?;
let amt = match status {
httparse::Status::Complete(amt) => amt,
httparse::Status::Partial => return Ok(None),
};
let toslice = |a: &[u8]| {
let start = a.as_ptr() as usize - src.as_ptr() as usize;
assert!(start < src.len());
(start, start + a.len())
};
for (i, header) in r.headers.iter().enumerate() {
let k = toslice(header.name.as_bytes());
let v = toslice(header.value);
headers[i] = Some((k, v));
}
(
toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt,
)
};
if version != 1 {
return Err(io::Error::new(
io::ErrorKind::Other,
"only HTTP/1.1 accepted",
));
}
let data = src.split_to(amt).freeze();
let mut ret = Request::builder();
ret = ret.method(&data[method.0..method.1]);
let s = data.slice(path.0..path.1);
let s = unsafe { String::from_utf8_unchecked(Vec::from(s.as_ref())) };
ret = ret.uri(s);
ret = ret.version(http::Version::HTTP_11);
for header in headers.iter() {
let (k, v) = match *header {
Some((ref k, ref v)) => (k, v),
None => break,
};
let value = HeaderValue::from_bytes(data.slice(v.0..v.1).as_ref())
.map_err(|_| io::Error::new(io::ErrorKind::Other, "header decode error"))?;
ret = ret.header(&data[k.0..k.1], value);
}
let req = ret
.body(())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
Ok(Some(req))
}
}
mod date {
use std::cell::RefCell;
use std::fmt::{self, Write};
use std::str;
use time::{self, Duration};
pub struct Now(());
/// Returns a struct, which when formatted, renders an appropriate `Date`
/// header value.
pub fn now() -> Now {
Now(())
}
// Gee Alex, doesn't this seem like premature optimization. Well you see
// there Billy, you're absolutely correct! If your server is *bottlenecked*
// on rendering the `Date` header, well then boy do I have news for you, you
// don't need this optimization.
//
// In all seriousness, though, a simple "hello world" benchmark which just
// sends back literally "hello world" with standard headers actually is
// bottlenecked on rendering a date into a byte buffer. Since it was at the
// top of a profile, and this was done for some competitive benchmarks, this
// module was written.
//
// Just to be clear, though, I was not intending on doing this because it
// really does seem kinda absurd, but it was done by someone else [1], so I
// blame them! :)
//
// [1]: https://github.com/rapidoid/rapidoid/blob/f1c55c0555007e986b5d069fe1086e6d09933f7b/rapidoid-commons/src/main/java/org/rapidoid/commons/Dates.java#L48-L66
struct LastRenderedNow {
bytes: [u8; 128],
amt: usize,
next_update: time::Timespec,
}
thread_local!(static LAST: RefCell<LastRenderedNow> = RefCell::new(LastRenderedNow {
bytes: [0; 128],
amt: 0,
next_update: time::Timespec::new(0, 0),
}));
impl fmt::Display for Now {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
LAST.with(|cache| {
let mut cache = cache.borrow_mut();
let now = time::get_time();
if now >= cache.next_update {
cache.update(now);
}
f.write_str(cache.buffer())
})
}
}
impl LastRenderedNow {
fn buffer(&self) -> &str {
str::from_utf8(&self.bytes[..self.amt]).unwrap()
}
fn update(&mut self, now: time::Timespec) {
self.amt = 0;
write!(LocalBuffer(self), "{}", time::at(now).rfc822()).unwrap();
self.next_update = now + Duration::seconds(1);
self.next_update.nsec = 0;
}
}
struct LocalBuffer<'a>(&'a mut LastRenderedNow);
impl fmt::Write for LocalBuffer<'_> {
fn write_str(&mut self, s: &str) -> fmt::Result {
let start = self.0.amt;
let end = start + s.len();
self.0.bytes[start..end].copy_from_slice(s.as_bytes());
self.0.amt += s.len();
Ok(())
}
}
}
-72
View File
@@ -1,72 +0,0 @@
//! A UDP client that just sends everything it gets via `stdio` in a single datagram, and then
//! waits for a reply.
//!
//! For the reasons of simplicity data from `stdio` is read until `EOF` in a blocking manner.
//!
//! You can test this out by running an echo server:
//!
//! ```
//! $ cargo run --example echo-udp -- 127.0.0.1:8080
//! ```
//!
//! and running the client in another terminal:
//!
//! ```
//! $ cargo run --example udp-client
//! ```
//!
//! You can optionally provide any custom endpoint address for the client:
//!
//! ```
//! $ cargo run --example udp-client -- 127.0.0.1:8080
//! ```
//!
//! Don't forget to pass `EOF` to the standard input of the client!
//!
//! Please mind that since the UDP protocol doesn't have any capabilities to detect a broken
//! connection the server needs to be run first, otherwise the client will block forever.
#![warn(rust_2018_idioms)]
use std::env;
use std::error::Error;
use std::io::{stdin, Read};
use std::net::SocketAddr;
use tokio::net::UdpSocket;
fn get_stdin_data() -> Result<Vec<u8>, Box<dyn std::error::Error>> {
let mut buf = Vec::new();
stdin().read_to_end(&mut buf)?;
Ok(buf)
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let remote_addr: SocketAddr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8080".into())
.parse()?;
// We use port 0 to let the operating system allocate an available port for us.
let local_addr: SocketAddr = if remote_addr.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
}
.parse()?;
let socket = UdpSocket::bind(local_addr).await?;
const MAX_DATAGRAM_SIZE: usize = 65_507;
socket.connect(&remote_addr).await?;
let data = get_stdin_data()?;
socket.send(&data).await?;
let mut data = vec![0u8; MAX_DATAGRAM_SIZE];
let len = socket.recv(&mut data).await?;
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
);
Ok(())
}
-80
View File
@@ -1,80 +0,0 @@
//! This example leverages `BytesCodec` to create a UDP client and server which
//! speak a custom protocol.
//!
//! Here we're using the codec from `tokio-codec` to convert a UDP socket to a stream of
//! client messages. These messages are then processed and returned back as a
//! new message with a new destination. Overall, we then use this to construct a
//! "ping pong" pair where two sockets are sending messages back and forth.
#![warn(rust_2018_idioms)]
use tokio::net::UdpSocket;
use tokio::stream::StreamExt;
use tokio::{io, time};
use tokio_util::codec::BytesCodec;
use tokio_util::udp::UdpFramed;
use bytes::Bytes;
use futures::{FutureExt, SinkExt};
use std::env;
use std::error::Error;
use std::net::SocketAddr;
use std::time::Duration;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:0".to_string());
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr).await?;
let b = UdpSocket::bind(&addr).await?;
let b_addr = b.local_addr()?;
let mut a = UdpFramed::new(a, BytesCodec::new());
let mut b = UdpFramed::new(b, BytesCodec::new());
// Start off by sending a ping from a to b, afterwards we just print out
// what they send us and continually send pings
let a = ping(&mut a, b_addr);
// The second client we have will receive the pings from `a` and then send
// back pongs.
let b = pong(&mut b);
// Run both futures simultaneously of `a` and `b` sending messages back and forth.
match futures::future::try_join(a, b).await {
Err(e) => println!("an error occurred; error = {:?}", e),
_ => println!("done!"),
}
Ok(())
}
async fn ping(socket: &mut UdpFramed<BytesCodec>, b_addr: SocketAddr) -> Result<(), io::Error> {
socket.send((Bytes::from(&b"PING"[..]), b_addr)).await?;
for _ in 0..4usize {
let (bytes, addr) = socket.next().map(|e| e.unwrap()).await?;
println!("[a] recv: {}", String::from_utf8_lossy(&bytes));
socket.send((Bytes::from(&b"PING"[..]), addr)).await?;
}
Ok(())
}
async fn pong(socket: &mut UdpFramed<BytesCodec>) -> Result<(), io::Error> {
let timeout = Duration::from_millis(200);
while let Ok(Some(Ok((bytes, addr)))) = time::timeout(timeout, socket.next()).await {
println!("[b] recv: {}", String::from_utf8_lossy(&bytes));
socket.send((Bytes::from(&b"PONG"[..]), addr)).await?;
}
Ok(())
}
+63
View File
@@ -0,0 +1,63 @@
//! Asynchronous signal handling for Tokio
//!
//! This crate implements asynchronous signal handling for Tokio, an
//! asynchronous I/O framework in Rust. The primary type exported from this
//! crate, `unix::Signal`, allows listening for arbitrary signals on Unix
//! platforms, receiving them in an asynchronous fashion.
//!
//! Note that signal handling is in general a very tricky topic and should be
//! used with great care. This crate attempts to implement 'best practice' for
//! signal handling, but it should be evaluated for your own applications' needs
//! to see if it's suitable.
//!
//! The are some fundamental limitations of this crate documented on the
//! `Signal` structure as well.
//!
//! > **Note**: This crate compiles on Windows, but currently contains no
//! > bindings. Windows does not have signals like Unix does, but it
//! > does have a way to receive ctrl-c notifications at the console.
//! > It's planned that this will be bound and exported outside the
//! > `unix` module in the future!
#![doc(html_root_url = "https://docs.rs/tokio-signal/0.1")]
#![deny(missing_docs)]
#[macro_use]
extern crate futures;
extern crate tokio_core;
use futures::Future;
use futures::stream::Stream;
use tokio_core::reactor::Handle;
use tokio_core::io::{IoStream, IoFuture};
pub mod unix;
pub mod windows;
/// Creates a stream which receives "ctrl-c" notifications sent to a process.
///
/// In general signals are handled very differently across Unix and Windows, but
/// this is somewhat cross platform in terms of how it can be handled. A ctrl-c
/// event to a console process can be represented as a stream for both Windows
/// and Unix.
///
/// This function receives a `Handle` to an event loop and returns a future
/// which when resolves yields a stream receiving all signal events. Note that
/// there are a number of caveats listening for signals, and you may wish to
/// read up on the documentation in the `unix` or `windows` module to take a
/// peek.
pub fn ctrl_c(handle: &Handle) -> IoFuture<IoStream<()>> {
return ctrl_c_imp(handle);
#[cfg(unix)]
fn ctrl_c_imp(handle: &Handle) -> IoFuture<IoStream<()>> {
unix::Signal::new(unix::libc::SIGINT, handle).map(|x| {
x.map(|_| ()).boxed()
}).boxed()
}
#[cfg(windows)]
fn ctrl_c_imp(handle: &Handle) -> IoFuture<IoStream<()>> {
windows::Event::ctrl_c(handle).map(|x| x.boxed()).boxed()
}
}
+393
View File
@@ -0,0 +1,393 @@
//! Unix-specific types for signal handling.
//!
//! This module is only defined on Unix platforms and contains the primary
//! `Signal` type for receiving notifications of signals.
#![cfg(unix)]
pub extern crate libc;
extern crate mio;
extern crate mio_uds;
use std::cell::UnsafeCell;
use std::collections::HashSet;
use std::io::prelude::*;
use std::io;
use std::mem;
use std::os::unix::prelude::*;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering, ATOMIC_USIZE_INIT};
use std::sync::{Mutex, Once, ONCE_INIT};
use futures::future;
use futures::sync::mpsc::{Receiver, Sender, channel};
use futures::{Async, AsyncSink, Future};
use futures::{Sink, Stream, Poll};
use self::libc::c_int;
use self::mio::Poll as MioPoll;
use self::mio::unix::EventedFd;
use self::mio::{Evented, Token, Ready, PollOpt};
use self::mio_uds::UnixStream;
use tokio_core::io::IoFuture;
use tokio_core::reactor::{Handle, CoreId, PollEvented};
pub use self::libc::{SIGINT, SIGTERM, SIGUSR1, SIGUSR2};
pub use self::libc::{SIGHUP, SIGQUIT, SIGPIPE, SIGALRM, SIGTRAP};
// Number of different unix signals
const SIGNUM: usize = 32;
struct SignalInfo {
pending: AtomicBool,
// The ones interested in this signal
recipients: Mutex<Vec<(usize, Sender<c_int>)>>,
init: Once,
initialized: UnsafeCell<bool>,
prev: UnsafeCell<libc::sigaction>,
}
struct Globals {
sender: UnixStream,
receiver: UnixStream,
signals: [SignalInfo; SIGNUM],
drivers: Mutex<HashSet<CoreId>>,
}
impl Default for SignalInfo {
fn default() -> SignalInfo {
SignalInfo {
pending: AtomicBool::new(false),
init: ONCE_INIT,
initialized: UnsafeCell::new(false),
recipients: Mutex::new(Vec::new()),
prev: UnsafeCell::new(unsafe { mem::zeroed() }),
}
}
}
static mut GLOBALS: *mut Globals = 0 as *mut Globals;
fn globals() -> &'static Globals {
static INIT: Once = ONCE_INIT;
unsafe {
INIT.call_once(|| {
let (receiver, sender) = UnixStream::pair().unwrap();
let globals = Globals {
sender: sender,
receiver: receiver,
signals: Default::default(),
drivers: Mutex::new(HashSet::new()),
};
GLOBALS = Box::into_raw(Box::new(globals));
});
&*GLOBALS
}
}
/// Our global signal handler for all signals registered by this module.
///
/// The purpose of this signal handler is to primarily:
///
/// 1. Flag that our specific signal was received (e.g. store an atomic flag)
/// 2. Wake up driver tasks by writing a byte to a pipe
///
/// Those two operations shoudl both be async-signal safe. After that's done we
/// just try to call a previous signal handler, if any, to be "good denizens of
/// the internet"
extern fn handler(signum: c_int,
info: *mut libc::siginfo_t,
ptr: *mut libc::c_void) {
type FnSigaction = extern fn(c_int, *mut libc::siginfo_t, *mut libc::c_void);
type FnHandler = extern fn(c_int);
unsafe {
let slot = match (*GLOBALS).signals.get(signum as usize) {
Some(slot) => slot,
None => return,
};
slot.pending.store(true, Ordering::SeqCst);
// Send a wakeup, ignore any errors (anything reasonably possible is
// full pipe and then it will wake up anyway).
drop((*GLOBALS).sender.write(&[1]));
let fnptr = (*slot.prev.get()).sa_sigaction;
if fnptr == 0 || fnptr == libc::SIG_DFL || fnptr == libc::SIG_IGN {
return
}
if (*slot.prev.get()).sa_flags & libc::SA_SIGINFO == 0 {
let action = mem::transmute::<usize, FnHandler>(fnptr);
action(signum)
} else {
let action = mem::transmute::<usize, FnSigaction>(fnptr);
action(signum, info, ptr)
}
}
}
/// Enable this module to receive signal notifications for the `signal`
/// provided.
///
/// This will register the signal handler if it hasn't already been registered,
/// returning any error along the way if that fails.
fn signal_enable(signal: c_int) -> io::Result<()> {
let siginfo = match globals().signals.get(signal as usize) {
Some(slot) => slot,
None => {
return Err(io::Error::new(io::ErrorKind::Other, "signal too large"))
}
};
unsafe {
let mut err = None;
siginfo.init.call_once(|| {
let mut new: libc::sigaction = mem::zeroed();
new.sa_sigaction = handler as usize;
new.sa_flags = libc::SA_RESTART |
libc::SA_SIGINFO |
libc::SA_NOCLDSTOP;
if libc::sigaction(signal, &new, &mut *siginfo.prev.get()) != 0 {
err = Some(io::Error::last_os_error());
} else {
*siginfo.initialized.get() = true;
}
});
if let Some(err) = err {
return Err(err)
}
if *siginfo.initialized.get() {
Ok(())
} else {
Err(io::Error::new(io::ErrorKind::Other,
"failed to register signal handler"))
}
}
}
/// A helper struct to register our global receiving end of the signal pipe on
/// multiple event loops.
///
/// This structure represents registering the receiving end on all event loops,
/// and uses `EventedFd` in mio to do so. It's stored in each driver task and is
/// used to read data and register interest in new signals coming in.
struct EventedReceiver;
impl Evented for EventedReceiver {
fn register(&self, poll: &MioPoll, token: Token, events: Ready, opts: PollOpt) -> io::Result<()> {
let fd = globals().receiver.as_raw_fd();
EventedFd(&fd).register(poll, token, events, opts)
}
fn reregister(&self, poll: &MioPoll, token: Token, events: Ready, opts: PollOpt) -> io::Result<()> {
let fd = globals().receiver.as_raw_fd();
EventedFd(&fd).reregister(poll, token, events, opts)
}
fn deregister(&self, poll: &MioPoll) -> io::Result<()> {
let fd = globals().receiver.as_raw_fd();
EventedFd(&fd).deregister(poll)
}
}
impl Read for EventedReceiver {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
(&globals().receiver).read(buf)
}
}
struct Driver {
id: CoreId,
wakeup: PollEvented<EventedReceiver>,
}
impl Future for Driver {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
// Drain the data from the pipe and maintain interest in getting more
let any_wakeup = self.drain();
if any_wakeup {
self.broadcast();
}
// This task just lives until the end of the event loop
Ok(Async::NotReady)
}
}
impl Drop for Driver {
fn drop(&mut self) {
let mut drivers = globals().drivers.lock().unwrap();
drivers.remove(&self.id);
}
}
impl Driver {
fn new(handle: &Handle) -> io::Result<Driver> {
Ok(Driver {
id: handle.id(),
wakeup: try!(PollEvented::new(EventedReceiver, handle)),
})
}
/// Drain all data in the global receiver, returning whether data was to be
/// had.
///
/// If this function returns `true` then some signal has been received since
/// we last checked, otherwise `false` indicates that no signal has been
/// received.
fn drain(&mut self) -> bool {
let mut received = false;
loop {
match self.wakeup.read(&mut [0; 128]) {
Ok(0) => panic!("EOF on self-pipe"),
Ok(_) => received = true,
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => break,
Err(e) => panic!("Bad read on self-pipe: {}", e),
}
}
received
}
/// Go through all the signals and broadcast everything.
///
/// Driver tasks wake up for *any* signal and simply process all globally
/// registered signal streams, so each task is sort of cooperatively working
/// for all the rest as well.
fn broadcast(&self) {
for (sig, slot) in globals().signals.iter().enumerate() {
// Any signal of this kind arrived since we checked last?
if !slot.pending.swap(false, Ordering::SeqCst) {
continue
}
let signum = sig as c_int;
let mut recipients = slot.recipients.lock().unwrap();
// Notify all waiters on this signal that the signal has been
// received. If we can't push a message into the queue then we don't
// worry about it as everything is coalesced anyway. If the channel
// has gone away then we can remove that slot.
for i in (0..recipients.len()).rev() {
// TODO: This thing probably generates unnecessary wakups of
// this task when `NotReady` is received because we don't
// actually want to get woken up to continue sending a
// message. Let's optimise it later on though, as we know
// this works.
match recipients[i].1.start_send(signum) {
Ok(AsyncSink::Ready) => {}
Ok(AsyncSink::NotReady(_)) => {}
Err(_) => { recipients.swap_remove(i); }
}
}
}
}
}
/// An implementation of `Stream` for receiving a particular type of signal.
///
/// This structure implements the `Stream` trait and represents notifications
/// of the current process receiving a particular signal. The signal being
/// listened for is passed to `Signal::new`, and the same signal number is then
/// yielded as each element for the stream.
///
/// In general signal handling on Unix is a pretty tricky topic, and this
/// structure is no exception! There are some important limitations to keep in
/// mind when using `Signal` streams:
///
/// * Signals handling in Unix already necessitates coalescing signals
/// together sometimes. This `Signal` stream is also no exception here in
/// that it will also coalesce signals. That is, even if the signal handler
/// for this process runs multiple times, the `Signal` stream may only return
/// one signal notification. Specifically, before `poll` is called, all
/// signal notifications are coalesced into one item returned from `poll`.
/// Once `poll` has been called, however, a further signal is guaranteed to
/// be yielded as an item.
///
/// Put another way, any element pulled off the returned stream corresponds to
/// *at least one* signal, but possibly more.
///
/// * Signal handling in general is relatively inefficient. Although some
/// improvements are possible in this crate, it's recommended to not plan on
/// having millions of signal channels open.
///
/// * Currently the "driver task" to process incoming signals never exits. This
/// driver task runs in the background of the event loop provided, and
/// in general you shouldn't need to worry about it.
///
/// If you've got any questions about this feel free to open an issue on the
/// repo, though, as I'd love to chat about this! In other words, I'd love to
/// alleviate some of these limitations if possible!
pub struct Signal {
signal: c_int,
token: usize,
rx: Receiver<c_int>,
}
impl Signal {
/// Creates a new stream which will receive notifications when the current
/// process receives the signal `signal`.
///
/// This function will create a new stream which may be based on the
/// event loop handle provided. This function returns a future which will
/// then resolve to the signal stream, if successful.
///
/// The `Signal` stream is an infinite stream which will receive
/// notifications whenever a signal is received. More documentation can be
/// found on `Signal` itself, but to reiterate:
///
/// * Signals may be coalesced beyond what the kernel already does.
/// * Once a signal handler is registered with the process the underlying
/// libc signal handler is never unregistered.
///
/// A `Signal` stream can be created for a particular signal number
/// multiple times. When a signal is received then all the associated
/// channels will receive the signal notification.
pub fn new(signal: c_int, handle: &Handle) -> IoFuture<Signal> {
static TOKENS: AtomicUsize = ATOMIC_USIZE_INIT;
let result = (|| {
// Turn the signal delivery on once we are ready for it
try!(signal_enable(signal));
// Ensure there's a driver for our associated event loop processing
// signals.
let id = handle.id();
let mut drivers = globals().drivers.lock().unwrap();
if !drivers.contains(&id) {
handle.spawn(try!(Driver::new(handle)));
drivers.insert(id);
}
drop(drivers);
// One wakeup in a queue is enough, no need for us to buffer up any
// more.
let (tx, rx) = channel(1);
let token = TOKENS.fetch_add(1, Ordering::SeqCst);
let idx = signal as usize;
globals().signals[idx].recipients.lock().unwrap().push((token, tx));
Ok(Signal {
rx: rx,
token: token,
signal: signal,
})
})();
future::result(result).boxed()
}
}
impl Stream for Signal {
type Item = c_int;
type Error = io::Error;
fn poll(&mut self) -> Poll<Option<c_int>, io::Error> {
// receivers don't generate errors
self.rx.poll().map_err(|_| panic!())
}
}
impl Drop for Signal {
fn drop(&mut self) {
let idx = self.signal as usize;
let mut list = globals().signals[idx].recipients.lock().unwrap();
list.retain(|pair| pair.0 != self.token);
}
}
+294
View File
@@ -0,0 +1,294 @@
//! Windows-specific types for signal handling.
//!
//! This module is only defined on Windows and contains the primary `Event` type
//! for receiving notifications of events. These events are listened for via the
//! `SetConsoleCtrlHandler` function which receives events of the type
//! `CTRL_C_EVENT` and `CTRL_BREAK_EVENT`
#![cfg(windows)]
extern crate kernel32;
extern crate mio;
extern crate winapi;
use std::cell::RefCell;
use std::io;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Once, ONCE_INIT};
use futures::future;
use futures::stream::Fuse;
use futures::sync::mpsc;
use futures::sync::oneshot;
use futures::{Future, IntoFuture, Poll, Async, Stream};
use tokio_core::io::IoFuture;
use tokio_core::reactor::{PollEvented, Handle};
static INIT: Once = ONCE_INIT;
static mut GLOBAL_STATE: *mut GlobalState = 0 as *mut _;
/// Stream of events discovered via `SetConsoleCtrlHandler`.
///
/// This structure can be used to listen for events of the type `CTRL_C_EVENT`
/// and `CTRL_BREAK_EVENT`. The `Stream` trait is implemented for this struct
/// and will resolve for each notification received by the process. Note that
/// there are few limitations with this as well:
///
/// * A notification to this process notifies *all* `Event` streams for that
/// event type.
/// * Notifications to an `Event` stream **are coalesced** if they aren't
/// processed quickly enough. This means that if two notifications are
/// received back-to-back, then the stream may only receive one item about the
/// two notifications.
pub struct Event {
reg: PollEvented<MyRegistration>,
_finished: oneshot::Sender<()>,
}
struct GlobalState {
ready: mio::SetReadiness,
tx: mpsc::UnboundedSender<Message>,
ctrl_c: GlobalEventState,
ctrl_break: GlobalEventState,
}
struct GlobalEventState {
ready: AtomicBool,
}
enum Message {
NewEvent(winapi::DWORD, oneshot::Sender<io::Result<Event>>),
}
struct DriverTask {
handle: Handle,
reg: PollEvented<MyRegistration>,
rx: Fuse<mpsc::UnboundedReceiver<Message>>,
ctrl_c: EventState,
ctrl_break: EventState,
}
struct EventState {
tasks: Vec<(RefCell<oneshot::Receiver<()>>, mio::SetReadiness)>,
}
impl Event {
/// Creates a new stream listening for the `CTRL_C_EVENT` events.
///
/// This function will register a handler via `SetConsoleCtrlHandler` and
/// deliver notifications to the returned stream.
pub fn ctrl_c(handle: &Handle) -> IoFuture<Event> {
Event::new(winapi::CTRL_C_EVENT, handle)
}
/// Creates a new stream listening for the `CTRL_BREAK_EVENT` events.
///
/// This function will register a handler via `SetConsoleCtrlHandler` and
/// deliver notifications to the returned stream.
pub fn ctrl_break(handle: &Handle) -> IoFuture<Event> {
Event::new(winapi::CTRL_BREAK_EVENT, handle)
}
fn new(signum: winapi::DWORD, handle: &Handle) -> IoFuture<Event> {
let mut init = None;
INIT.call_once(|| {
init = Some(global_init(handle));
});
let new_signal = future::lazy(move || {
let (tx, rx) = oneshot::channel();
let msg = Message::NewEvent(signum, tx);
let res = unsafe {
(*GLOBAL_STATE).tx.clone().send(msg)
};
res.expect("failed to request a new signal stream, did the \
first event loop go away?");
rx.then(|r| r.unwrap())
});
match init {
Some(init) => init.into_future().and_then(|()| new_signal).boxed(),
None => new_signal.boxed(),
}
}
}
impl Stream for Event {
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<Option<()>, io::Error> {
if !self.reg.poll_read().is_ready() {
return Ok(Async::NotReady)
}
self.reg.need_read();
self.reg.get_ref()
.inner.borrow()
.as_ref().unwrap().1
.set_readiness(mio::Ready::none())
.expect("failed to set readiness");
Ok(Async::Ready(Some(())))
}
}
fn global_init(handle: &Handle) -> io::Result<()> {
let (tx, rx) = mpsc::unbounded();
let reg = MyRegistration { inner: RefCell::new(None) };
let reg = try!(PollEvented::new(reg, handle));
let ready = reg.get_ref().inner.borrow().as_ref().unwrap().1.clone();
unsafe {
let state = Box::new(GlobalState {
ready: ready,
ctrl_c: GlobalEventState { ready: AtomicBool::new(false) },
ctrl_break: GlobalEventState { ready: AtomicBool::new(false) },
tx: tx,
});
GLOBAL_STATE = Box::into_raw(state);
let rc = kernel32::SetConsoleCtrlHandler(Some(handler), winapi::TRUE);
if rc == 0 {
Box::from_raw(GLOBAL_STATE);
GLOBAL_STATE = 0 as *mut _;
return Err(io::Error::last_os_error())
}
handle.spawn(DriverTask {
handle: handle.clone(),
rx: rx.fuse(),
reg: reg,
ctrl_c: EventState { tasks: Vec::new() },
ctrl_break: EventState { tasks: Vec::new() },
});
Ok(())
}
}
impl Future for DriverTask {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.check_event_drops();
self.check_messages();
self.check_events();
// TODO: when to finish this task?
Ok(Async::NotReady)
}
}
impl DriverTask {
fn check_event_drops(&mut self) {
self.ctrl_c.tasks.retain(|task| {
!task.0.borrow_mut().poll().is_err()
});
self.ctrl_break.tasks.retain(|task| {
!task.0.borrow_mut().poll().is_err()
});
}
fn check_messages(&mut self) {
loop {
// Acquire the next message
let message = match self.rx.poll().unwrap() {
Async::Ready(Some(e)) => e,
Async::Ready(None) |
Async::NotReady => break,
};
let (sig, complete) = match message {
Message::NewEvent(sig, complete) => (sig, complete),
};
let event = if sig == winapi::CTRL_C_EVENT {
&mut self.ctrl_c
} else {
&mut self.ctrl_break
};
// Acquire the (registration, set_readiness) pair by... assuming
// we're on the event loop (true because of the spawn above).
let reg = MyRegistration { inner: RefCell::new(None) };
let reg = match PollEvented::new(reg, &self.handle) {
Ok(reg) => reg,
Err(e) => {
complete.complete(Err(e));
continue
}
};
// Create the `Event` to pass back and then also keep a handle to
// the `SetReadiness` for ourselves internally.
let (tx, rx) = oneshot::channel();
let ready = reg.get_ref().inner.borrow_mut().as_mut().unwrap().1.clone();
complete.complete(Ok(Event {
reg: reg,
_finished: tx,
}));
event.tasks.push((RefCell::new(rx), ready));
}
}
fn check_events(&mut self) {
if self.reg.poll_read().is_not_ready() {
return
}
self.reg.need_read();
self.reg.get_ref().inner.borrow().as_ref().unwrap()
.1.set_readiness(mio::Ready::none()).unwrap();
if unsafe { (*GLOBAL_STATE).ctrl_c.ready.swap(false, Ordering::SeqCst) } {
for task in self.ctrl_c.tasks.iter() {
task.1.set_readiness(mio::Ready::readable()).unwrap();
}
}
if unsafe { (*GLOBAL_STATE).ctrl_break.ready.swap(false, Ordering::SeqCst) } {
for task in self.ctrl_break.tasks.iter() {
task.1.set_readiness(mio::Ready::readable()).unwrap();
}
}
}
}
unsafe extern "system" fn handler(ty: winapi::DWORD) -> winapi::BOOL {
let event = match ty {
winapi::CTRL_C_EVENT => &(*GLOBAL_STATE).ctrl_c,
winapi::CTRL_BREAK_EVENT => &(*GLOBAL_STATE).ctrl_break,
_ => return winapi::FALSE
};
if event.ready.swap(true, Ordering::SeqCst) {
winapi::FALSE
} else {
drop((*GLOBAL_STATE).ready.set_readiness(mio::Ready::readable()));
// TODO: this will report that we handled a CTRL_BREAK_EVENT when in
// fact we may not have any streams actually created for that
// event.
winapi::TRUE
}
}
struct MyRegistration {
inner: RefCell<Option<(mio::Registration, mio::SetReadiness)>>,
}
impl mio::Evented for MyRegistration {
fn register(&self,
poll: &mio::Poll,
token: mio::Token,
events: mio::Ready,
opts: mio::PollOpt) -> io::Result<()> {
let reg = mio::Registration::new(poll, token, events, opts);
*self.inner.borrow_mut() = Some(reg);
Ok(())
}
fn reregister(&self,
_poll: &mio::Poll,
_token: mio::Token,
_events: mio::Ready,
_opts: mio::PollOpt) -> io::Result<()> {
Ok(())
}
fn deregister(&self, _poll: &mio::Poll) -> io::Result<()> {
Ok(())
}
}
-14
View File
@@ -1,14 +0,0 @@
[package]
name = "stress-test"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
tokio = {path = "../tokio/", features = ["full"]}
[dev-dependencies]
rand = "0.7.3"
-58
View File
@@ -1,58 +0,0 @@
//! Simple TCP echo server to check memory leaks using Valgrind.
use std::{thread::sleep, time::Duration};
use tokio::{
io::{AsyncReadExt, AsyncWriteExt},
net::{TcpListener, TcpSocket},
runtime::Builder,
sync::oneshot,
};
const TCP_ENDPOINT: &str = "127.0.0.1:8080";
const NUM_MSGS: usize = 100;
const MSG_SIZE: usize = 1024;
fn main() {
let rt = Builder::new_multi_thread().enable_io().build().unwrap();
let rt2 = Builder::new_multi_thread().enable_io().build().unwrap();
rt.spawn(async {
let listener = TcpListener::bind(TCP_ENDPOINT).await.unwrap();
let (mut socket, _) = listener.accept().await.unwrap();
let (mut rd, mut wr) = socket.split();
while tokio::io::copy(&mut rd, &mut wr).await.is_ok() {}
});
// wait a bit so that the listener binds.
sleep(Duration::from_millis(100));
// create a channel to let the main thread know that all the messages were sent and received.
let (tx, mut rx) = oneshot::channel();
rt2.spawn(async {
let addr = TCP_ENDPOINT.parse().unwrap();
let socket = TcpSocket::new_v4().unwrap();
let mut stream = socket.connect(addr).await.unwrap();
let mut buff = [0; MSG_SIZE];
for _ in 0..NUM_MSGS {
let one_mega_random_bytes: Vec<u8> =
(0..MSG_SIZE).map(|_| rand::random::<u8>()).collect();
stream
.write_all(one_mega_random_bytes.as_slice())
.await
.unwrap();
stream.read(&mut buff).await.unwrap();
}
tx.send(()).unwrap();
});
loop {
// check that we're done.
match rx.try_recv() {
Err(oneshot::error::TryRecvError::Empty) => (),
Err(oneshot::error::TryRecvError::Closed) => panic!("channel got closed..."),
Ok(()) => break,
}
}
}
-16
View File
@@ -1,16 +0,0 @@
[package]
name = "tests-build"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[features]
full = ["tokio/full"]
rt = ["tokio/rt", "tokio/macros"]
[dependencies]
tokio = { path = "../tokio", optional = true }
[dev-dependencies]
trybuild = "1.0"
-2
View File
@@ -1,2 +0,0 @@
Tests the various combination of feature flags. This is broken out to a separate
crate to work around limitations with cargo features.
-2
View File
@@ -1,2 +0,0 @@
#[cfg(feature = "tokio")]
pub use tokio;
@@ -1,6 +0,0 @@
use tests_build::tokio;
#[tokio::main]
async fn my_fn() {}
fn main() {}
@@ -1,7 +0,0 @@
error: The default runtime flavor is `multi_thread`, but the `rt-multi-thread` feature is disabled.
--> $DIR/macros_core_no_default.rs:3:1
|
3 | #[tokio::main]
| ^^^^^^^^^^^^^^
|
= note: this error originates in an attribute macro (in Nightly builds, run with -Z macro-backtrace for more info)
@@ -1,25 +0,0 @@
use tests_build::tokio;
#[tokio::main]
fn main_is_not_async() {}
#[tokio::main(foo)]
async fn main_attr_has_unknown_args() {}
#[tokio::main(threadpool::bar)]
async fn main_attr_has_path_args() {}
#[tokio::test]
fn test_is_not_async() {}
#[tokio::test]
async fn test_fn_has_args(_x: u8) {}
#[tokio::test(foo)]
async fn test_attr_has_args() {}
#[tokio::test]
#[test]
async fn test_has_second_test_attr() {}
fn main() {}
@@ -1,41 +0,0 @@
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:4:1
|
4 | fn main_is_not_async() {}
| ^^
error: Unknown attribute foo is specified; expected one of: `flavor`, `worker_threads`
--> $DIR/macros_invalid_input.rs:6:15
|
6 | #[tokio::main(foo)]
| ^^^
error: Must have specified ident
--> $DIR/macros_invalid_input.rs:9:15
|
9 | #[tokio::main(threadpool::bar)]
| ^^^^^^^^^^^^^^^
error: the async keyword is missing from the function declaration
--> $DIR/macros_invalid_input.rs:13:1
|
13 | fn test_is_not_async() {}
| ^^
error: the test function cannot accept arguments
--> $DIR/macros_invalid_input.rs:16:27
|
16 | async fn test_fn_has_args(_x: u8) {}
| ^^^^^^
error: Unknown attribute foo is specified; expected one of: `flavor`, `worker_threads`
--> $DIR/macros_invalid_input.rs:18:15
|
18 | #[tokio::test(foo)]
| ^^^
error: second test attribute is supplied
--> $DIR/macros_invalid_input.rs:22:1
|
22 | #[test]
| ^^^^^^^
-12
View File
@@ -1,12 +0,0 @@
#[test]
fn compile_fail_full() {
let t = trybuild::TestCases::new();
#[cfg(feature = "full")]
t.compile_fail("tests/fail/macros_invalid_input.rs");
#[cfg(all(feature = "rt", not(feature = "full")))]
t.compile_fail("tests/fail/macros_core_no_default.rs");
drop(t);
}
-28
View File
@@ -1,28 +0,0 @@
[package]
name = "tests-integration"
version = "0.1.0"
authors = ["Tokio Contributors <[email protected]>"]
edition = "2018"
publish = false
[features]
full = [
"macros",
"rt",
"rt-multi-thread",
"tokio/full",
"tokio-test"
]
macros = ["tokio/macros"]
sync = ["tokio/sync"]
rt = ["tokio/rt"]
rt-multi-thread = ["rt", "tokio/rt-multi-thread"]
[dependencies]
tokio = { path = "../tokio" }
tokio-test = { path = "../tokio-test", optional = true }
doc-comment = "0.3.1"
[dev-dependencies]
futures = { version = "0.3.0", features = ["async-await"] }
-1
View File
@@ -1 +0,0 @@
Tests that require additional components than just the `tokio` crate.
-20
View File
@@ -1,20 +0,0 @@
//! A cat-like utility that can be used as a subprocess to test I/O
//! stream communication.
use std::io;
use std::io::Write;
fn main() {
let stdin = io::stdin();
let mut stdout = io::stdout();
let mut line = String::new();
loop {
line.clear();
stdin.read_line(&mut line).unwrap();
if line.is_empty() {
break;
}
stdout.write_all(line.as_bytes()).unwrap();
}
stdout.flush().unwrap();
}
-2
View File
@@ -1,2 +0,0 @@
#[cfg(feature = "full")]
doc_comment::doc_comment!(include_str!("../../README.md"));
-28
View File
@@ -1,28 +0,0 @@
#![cfg(all(feature = "macros", feature = "rt"))]
#[tokio::main]
async fn basic_main() -> usize {
1
}
#[tokio::main]
async fn generic_fun<T: Default>() -> T {
T::default()
}
#[tokio::main]
async fn spawning() -> usize {
let join = tokio::spawn(async { 1 });
join.await.unwrap()
}
#[test]
fn main_with_spawn() {
assert_eq!(1, spawning());
}
#[test]
fn shell() {
assert_eq!(1, basic_main());
assert_eq!(bool::default(), generic_fun::<bool>())
}
-12
View File
@@ -1,12 +0,0 @@
use futures::executor::block_on;
async fn my_async_fn() {}
#[test]
fn pin() {
block_on(async {
let future = my_async_fn();
tokio::pin!(future);
(&mut future).await
});
}
-33
View File
@@ -1,33 +0,0 @@
#![cfg(feature = "macros")]
use futures::channel::oneshot;
use futures::executor::block_on;
use std::thread;
#[test]
fn join_with_select() {
block_on(async {
let (tx1, mut rx1) = oneshot::channel::<i32>();
let (tx2, mut rx2) = oneshot::channel::<i32>();
thread::spawn(move || {
tx1.send(123).unwrap();
tx2.send(456).unwrap();
});
let mut a = None;
let mut b = None;
while a.is_none() || b.is_none() {
tokio::select! {
v1 = (&mut rx1), if a.is_none() => a = Some(v1.unwrap()),
v2 = (&mut rx2), if b.is_none() => b = Some(v2.unwrap()),
}
}
let (a, b) = (a.unwrap(), b.unwrap());
assert_eq!(a, 123);
assert_eq!(b, 456);
});
}
-150
View File
@@ -1,150 +0,0 @@
#![warn(rust_2018_idioms)]
#![cfg(feature = "full")]
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::process::{Child, Command};
use tokio_test::assert_ok;
use futures::future::{self, FutureExt};
use std::env;
use std::io;
use std::process::{ExitStatus, Stdio};
fn cat() -> Command {
let mut me = env::current_exe().unwrap();
me.pop();
if me.ends_with("deps") {
me.pop();
}
me.push("test-cat");
let mut cmd = Command::new(me);
cmd.stdin(Stdio::piped()).stdout(Stdio::piped());
cmd
}
async fn feed_cat(mut cat: Child, n: usize) -> io::Result<ExitStatus> {
let mut stdin = cat.stdin.take().unwrap();
let stdout = cat.stdout.take().unwrap();
// Produce n lines on the child's stdout.
let write = async {
for i in 0..n {
let bytes = format!("line {}\n", i).into_bytes();
stdin.write_all(&bytes).await.unwrap();
}
drop(stdin);
};
let read = async {
let mut reader = BufReader::new(stdout).lines();
let mut num_lines = 0;
// Try to read `n + 1` lines, ensuring the last one is empty
// (i.e. EOF is reached after `n` lines.
loop {
let data = reader
.next_line()
.await
.unwrap_or_else(|_| Some(String::new()))
.expect("failed to read line");
let num_read = data.len();
let done = num_lines >= n;
match (done, num_read) {
(false, 0) => panic!("broken pipe"),
(true, n) if n != 0 => panic!("extraneous data"),
_ => {
let expected = format!("line {}", num_lines);
assert_eq!(expected, data);
}
};
num_lines += 1;
if num_lines >= n {
break;
}
}
};
// Compose reading and writing concurrently.
future::join3(write, read, cat.wait())
.map(|(_, _, status)| status)
.await
}
/// Check for the following properties when feeding stdin and
/// consuming stdout of a cat-like process:
///
/// - A number of lines that amounts to a number of bytes exceeding a
/// typical OS buffer size can be fed to the child without
/// deadlock. This tests that we also consume the stdout
/// concurrently; otherwise this would deadlock.
///
/// - We read the same lines from the child that we fed it.
///
/// - The child does produce EOF on stdout after the last line.
#[tokio::test]
async fn feed_a_lot() {
let child = cat().spawn().unwrap();
let status = feed_cat(child, 10000).await.unwrap();
assert_eq!(status.code(), Some(0));
}
#[tokio::test]
async fn wait_with_output_captures() {
let mut child = cat().spawn().unwrap();
let mut stdin = child.stdin.take().unwrap();
let write_bytes = b"1234";
let future = async {
stdin.write_all(write_bytes).await?;
drop(stdin);
let out = child.wait_with_output();
out.await
};
let output = future.await.unwrap();
assert!(output.status.success());
assert_eq!(output.stdout, write_bytes);
assert_eq!(output.stderr.len(), 0);
}
#[tokio::test]
async fn status_closes_any_pipes() {
// Cat will open a pipe between the parent and child.
// If `status_async` doesn't ensure the handles are closed,
// we would end up blocking forever (and time out).
let child = cat().status();
assert_ok!(child.await);
}
#[tokio::test]
async fn try_wait() {
let mut child = cat().spawn().unwrap();
let id = child.id().expect("missing id");
assert!(id > 0);
assert_eq!(None, assert_ok!(child.try_wait()));
// Drop the child's stdio handles so it can terminate
drop(child.stdin.take());
drop(child.stderr.take());
drop(child.stdout.take());
assert_ok!(child.wait().await);
// test that the `.try_wait()` method is fused just like the stdlib
assert!(assert_ok!(child.try_wait()).unwrap().success());
// Can't get id after process has exited
assert_eq!(child.id(), None);
}
+65
View File
@@ -0,0 +1,65 @@
#![cfg(unix)]
extern crate futures;
extern crate libc;
extern crate tokio_core;
extern crate tokio_signal;
use std::time::Duration;
use futures::Future;
use futures::stream::Stream;
use tokio_core::reactor::{Core, Timeout};
use tokio_signal::unix::Signal;
#[test]
fn simple() {
let mut lp = Core::new().unwrap();
let handle = lp.handle();
let signal = lp.run(Signal::new(libc::SIGUSR1, &handle)).unwrap();
unsafe {
assert_eq!(libc::kill(libc::getpid(), libc::SIGUSR1), 0);
}
lp.run(signal.into_future()).ok().unwrap();
}
#[test]
fn notify_both() {
let mut lp = Core::new().unwrap();
let handle = lp.handle();
let signal1 = lp.run(Signal::new(libc::SIGUSR2, &handle)).unwrap();
let signal2 = lp.run(Signal::new(libc::SIGUSR2, &handle)).unwrap();
unsafe {
assert_eq!(libc::kill(libc::getpid(), libc::SIGUSR2), 0);
}
lp.run(signal1.into_future().join(signal2.into_future())).ok().unwrap();
}
#[test]
fn drop_then_get_a_signal() {
let mut lp = Core::new().unwrap();
let handle = lp.handle();
let signal = lp.run(Signal::new(libc::SIGUSR1, &handle)).unwrap();
drop(signal);
unsafe {
assert_eq!(libc::kill(libc::getpid(), libc::SIGUSR1), 0);
}
let timeout = Timeout::new(Duration::from_millis(1), &lp.handle()).unwrap();
lp.run(timeout).unwrap();
}
#[test]
fn twice() {
let mut lp = Core::new().unwrap();
let handle = lp.handle();
let signal = lp.run(Signal::new(libc::SIGUSR1, &handle)).unwrap();
unsafe {
assert_eq!(libc::kill(libc::getpid(), libc::SIGUSR1), 0);
}
let (num, signal) = lp.run(signal.into_future()).ok().unwrap();
assert_eq!(num, Some(libc::SIGUSR1));
unsafe {
assert_eq!(libc::kill(libc::getpid(), libc::SIGUSR1), 0);
}
lp.run(signal.into_future()).ok().unwrap();
}
-62
View File
@@ -1,62 +0,0 @@
# 0.3.2 (December 19, 2020)
### Fixed
- fix outdated macro documentation ([#3180])
- add portability note to `tokio::main` ([#3199])
# 0.3.1 (October 25, 2020)
### Fixed
- fix incorrect docs regarding `max_threads` option ([#3038])
# 0.3.0 (October 15, 2020)
- Track `tokio` 0.3 release.
### Changed
- options are renamed to track `tokio` runtime builder fn names.
- `#[tokio::main]` macro requires `rt-multi-thread` when no `flavor` is specified.
# 0.2.5 (February 27, 2019)
### Fixed
- doc improvements ([#2225]).
# 0.2.4 (January 27, 2019)
### Fixed
- generics on `#[tokio::main]` function ([#2177]).
### Added
- support for `tokio::select!` ([#2152]).
# 0.2.3 (January 7, 2019)
### Fixed
- Revert breaking change.
# 0.2.2 (January 7, 2019)
### Added
- General refactoring and inclusion of additional runtime options ([#2022] and [#2038])
# 0.2.1 (December 18, 2019)
### Fixes
- inherit visibility when wrapping async fn ([#1954]).
# 0.2.0 (November 26, 2019)
- Initial release
[#1954]: https://github.com/tokio-rs/tokio/pull/1954
[#2022]: https://github.com/tokio-rs/tokio/pull/2022
[#2038]: https://github.com/tokio-rs/tokio/pull/2038
[#2152]: https://github.com/tokio-rs/tokio/pull/2152
[#2177]: https://github.com/tokio-rs/tokio/pull/2177
[#2225]: https://github.com/tokio-rs/tokio/pull/2225
[#3038]: https://github.com/tokio-rs/tokio/pull/3038
[#3180]: https://github.com/tokio-rs/tokio/pull/3180
[#3199]: https://github.com/tokio-rs/tokio/pull/3199
-36
View File
@@ -1,36 +0,0 @@
[package]
name = "tokio-macros"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - Update CHANGELOG.md.
# - Create "v0.3.x" git tag.
version = "0.3.2"
edition = "2018"
authors = ["Tokio Contributors <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-macros/0.3.2/tokio_macros"
description = """
Tokio's proc macros.
"""
categories = ["asynchronous"]
[lib]
proc-macro = true
[features]
[dependencies]
proc-macro2 = "1.0.7"
quote = "1"
syn = { version = "1.0.3", features = ["full"] }
[dev-dependencies]
tokio = { version = "0.3.0", path = "../tokio", features = ["full"] }
[package.metadata.docs.rs]
all-features = true
-47
View File
@@ -1,47 +0,0 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
The MIT License (MIT)
Copyright (c) 2019 Yoshua Wuyts
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-13
View File
@@ -1,13 +0,0 @@
# Tokio Macros
Procedural macros for use with Tokio
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-300
View File
@@ -1,300 +0,0 @@
use proc_macro::TokenStream;
use proc_macro2::Span;
use quote::quote;
use syn::spanned::Spanned;
#[derive(Clone, Copy, PartialEq)]
enum RuntimeFlavor {
CurrentThread,
Threaded,
}
impl RuntimeFlavor {
fn from_str(s: &str) -> Result<RuntimeFlavor, String> {
match s {
"current_thread" => Ok(RuntimeFlavor::CurrentThread),
"multi_thread" => Ok(RuntimeFlavor::Threaded),
"single_thread" => Err("The single threaded runtime flavor is called `current_thread`.".to_string()),
"basic_scheduler" => Err("The `basic_scheduler` runtime flavor has been renamed to `current_thread`.".to_string()),
"threaded_scheduler" => Err("The `threaded_scheduler` runtime flavor has been renamed to `multi_thread`.".to_string()),
_ => Err(format!("No such runtime flavor `{}`. The runtime flavors are `current_thread` and `multi_thread`.", s)),
}
}
}
struct FinalConfig {
flavor: RuntimeFlavor,
worker_threads: Option<usize>,
}
struct Configuration {
rt_multi_thread_available: bool,
default_flavor: RuntimeFlavor,
flavor: Option<RuntimeFlavor>,
worker_threads: Option<(usize, Span)>,
}
impl Configuration {
fn new(is_test: bool, rt_multi_thread: bool) -> Self {
Configuration {
rt_multi_thread_available: rt_multi_thread,
default_flavor: match is_test {
true => RuntimeFlavor::CurrentThread,
false => RuntimeFlavor::Threaded,
},
flavor: None,
worker_threads: None,
}
}
fn set_flavor(&mut self, runtime: syn::Lit, span: Span) -> Result<(), syn::Error> {
if self.flavor.is_some() {
return Err(syn::Error::new(span, "`flavor` set multiple times."));
}
let runtime_str = parse_string(runtime, span, "flavor")?;
let runtime =
RuntimeFlavor::from_str(&runtime_str).map_err(|err| syn::Error::new(span, err))?;
self.flavor = Some(runtime);
Ok(())
}
fn set_worker_threads(
&mut self,
worker_threads: syn::Lit,
span: Span,
) -> Result<(), syn::Error> {
if self.worker_threads.is_some() {
return Err(syn::Error::new(
span,
"`worker_threads` set multiple times.",
));
}
let worker_threads = parse_int(worker_threads, span, "worker_threads")?;
if worker_threads == 0 {
return Err(syn::Error::new(span, "`worker_threads` may not be 0."));
}
self.worker_threads = Some((worker_threads, span));
Ok(())
}
fn build(&self) -> Result<FinalConfig, syn::Error> {
let flavor = self.flavor.unwrap_or(self.default_flavor);
use RuntimeFlavor::*;
match (flavor, self.worker_threads) {
(CurrentThread, Some((_, worker_threads_span))) => Err(syn::Error::new(
worker_threads_span,
"The `worker_threads` option requires the `multi_thread` runtime flavor.",
)),
(CurrentThread, None) => Ok(FinalConfig {
flavor,
worker_threads: None,
}),
(Threaded, worker_threads) if self.rt_multi_thread_available => Ok(FinalConfig {
flavor,
worker_threads: worker_threads.map(|(val, _span)| val),
}),
(Threaded, _) => {
let msg = if self.flavor.is_none() {
"The default runtime flavor is `multi_thread`, but the `rt-multi-thread` feature is disabled."
} else {
"The runtime flavor `multi_thread` requires the `rt-multi-thread` feature."
};
Err(syn::Error::new(Span::call_site(), msg))
}
}
}
}
fn parse_int(int: syn::Lit, span: Span, field: &str) -> Result<usize, syn::Error> {
match int {
syn::Lit::Int(lit) => match lit.base10_parse::<usize>() {
Ok(value) => Ok(value),
Err(e) => Err(syn::Error::new(
span,
format!("Failed to parse {} as integer: {}", field, e),
)),
},
_ => Err(syn::Error::new(
span,
format!("Failed to parse {} as integer.", field),
)),
}
}
fn parse_string(int: syn::Lit, span: Span, field: &str) -> Result<String, syn::Error> {
match int {
syn::Lit::Str(s) => Ok(s.value()),
syn::Lit::Verbatim(s) => Ok(s.to_string()),
_ => Err(syn::Error::new(
span,
format!("Failed to parse {} as string.", field),
)),
}
}
fn parse_knobs(
mut input: syn::ItemFn,
args: syn::AttributeArgs,
is_test: bool,
rt_multi_thread: bool,
) -> Result<TokenStream, syn::Error> {
let sig = &mut input.sig;
let body = &input.block;
let attrs = &input.attrs;
let vis = input.vis;
if sig.asyncness.is_none() {
let msg = "the async keyword is missing from the function declaration";
return Err(syn::Error::new_spanned(sig.fn_token, msg));
}
sig.asyncness = None;
let macro_name = if is_test {
"tokio::test"
} else {
"tokio::main"
};
let mut config = Configuration::new(is_test, rt_multi_thread);
for arg in args {
match arg {
syn::NestedMeta::Meta(syn::Meta::NameValue(namevalue)) => {
let ident = namevalue.path.get_ident();
if ident.is_none() {
let msg = "Must have specified ident";
return Err(syn::Error::new_spanned(namevalue, msg));
}
match ident.unwrap().to_string().to_lowercase().as_str() {
"worker_threads" => {
config.set_worker_threads(namevalue.lit.clone(), namevalue.span())?;
}
"flavor" => {
config.set_flavor(namevalue.lit.clone(), namevalue.span())?;
}
"core_threads" => {
let msg = "Attribute `core_threads` is renamed to `worker_threads`";
return Err(syn::Error::new_spanned(namevalue, msg));
}
name => {
let msg = format!("Unknown attribute {} is specified; expected one of: `flavor`, `worker_threads`", name);
return Err(syn::Error::new_spanned(namevalue, msg));
}
}
}
syn::NestedMeta::Meta(syn::Meta::Path(path)) => {
let ident = path.get_ident();
if ident.is_none() {
let msg = "Must have specified ident";
return Err(syn::Error::new_spanned(path, msg));
}
let name = ident.unwrap().to_string().to_lowercase();
let msg = match name.as_str() {
"threaded_scheduler" | "multi_thread" => {
format!(
"Set the runtime flavor with #[{}(flavor = \"multi_thread\")].",
macro_name
)
}
"basic_scheduler" | "current_thread" | "single_threaded" => {
format!(
"Set the runtime flavor with #[{}(flavor = \"current_thread\")].",
macro_name
)
}
"flavor" | "worker_threads" => {
format!("The `{}` attribute requires an argument.", name)
}
name => {
format!("Unknown attribute {} is specified; expected one of: `flavor`, `worker_threads`", name)
}
};
return Err(syn::Error::new_spanned(path, msg));
}
other => {
return Err(syn::Error::new_spanned(
other,
"Unknown attribute inside the macro",
));
}
}
}
let config = config.build()?;
let mut rt = match config.flavor {
RuntimeFlavor::CurrentThread => quote! {
tokio::runtime::Builder::new_current_thread()
},
RuntimeFlavor::Threaded => quote! {
tokio::runtime::Builder::new_multi_thread()
},
};
if let Some(v) = config.worker_threads {
rt = quote! { #rt.worker_threads(#v) };
}
let header = {
if is_test {
quote! {
#[::core::prelude::v1::test]
}
} else {
quote! {}
}
};
let result = quote! {
#header
#(#attrs)*
#vis #sig {
#rt
.enable_all()
.build()
.unwrap()
.block_on(async { #body })
}
};
Ok(result.into())
}
#[cfg(not(test))] // Work around for rust-lang/rust#62127
pub(crate) fn main(args: TokenStream, item: TokenStream, rt_multi_thread: bool) -> TokenStream {
let input = syn::parse_macro_input!(item as syn::ItemFn);
let args = syn::parse_macro_input!(args as syn::AttributeArgs);
if input.sig.ident == "main" && !input.sig.inputs.is_empty() {
let msg = "the main function cannot accept arguments";
return syn::Error::new_spanned(&input.sig.ident, msg)
.to_compile_error()
.into();
}
parse_knobs(input, args, false, rt_multi_thread).unwrap_or_else(|e| e.to_compile_error().into())
}
pub(crate) fn test(args: TokenStream, item: TokenStream, rt_multi_thread: bool) -> TokenStream {
let input = syn::parse_macro_input!(item as syn::ItemFn);
let args = syn::parse_macro_input!(args as syn::AttributeArgs);
for attr in &input.attrs {
if attr.path.is_ident("test") {
let msg = "second test attribute is supplied";
return syn::Error::new_spanned(&attr, msg)
.to_compile_error()
.into();
}
}
if !input.sig.inputs.is_empty() {
let msg = "the test function cannot accept arguments";
return syn::Error::new_spanned(&input.sig.inputs, msg)
.to_compile_error()
.into();
}
parse_knobs(input, args, true, rt_multi_thread).unwrap_or_else(|e| e.to_compile_error().into())
}
-295
View File
@@ -1,295 +0,0 @@
#![doc(html_root_url = "https://docs.rs/tokio-macros/0.3.2")]
#![allow(clippy::needless_doctest_main)]
#![warn(
missing_debug_implementations,
missing_docs,
rust_2018_idioms,
unreachable_pub
)]
#![cfg_attr(docsrs, deny(broken_intra_doc_links))]
#![doc(test(
no_crate_inject,
attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))
))]
//! Macros for use with Tokio
// This `extern` is required for older `rustc` versions but newer `rustc`
// versions warn about the unused `extern crate`.
#[allow(unused_extern_crates)]
extern crate proc_macro;
mod entry;
mod select;
use proc_macro::TokenStream;
/// Marks async function to be executed by the selected runtime. This macro helps
/// set up a `Runtime` without requiring the user to use
/// [Runtime](../tokio/runtime/struct.Runtime.html) or
/// [Builder](../tokio/runtime/struct.Builder.html) directly.
///
/// Note: This macro is designed to be simplistic and targets applications that
/// do not require a complex setup. If the provided functionality is not
/// sufficient, you may be interested in using
/// [Builder](../tokio/runtime/struct.Builder.html), which provides a more
/// powerful interface.
///
/// Note: This macro can be used on any function and not just the `main`
/// function. Using it on a non-main function makes the function behave
/// as if it was synchronous by starting a new runtime each time it is called.
/// If the function is called often, it is preferable to create the runtime using
/// the runtime builder so the runtime can be reused across calls.
///
/// # Multi-threaded runtime
///
/// To use the multi-threaded runtime, the macro can be configured using
///
/// ```
/// #[tokio::main(flavor = "multi_thread", worker_threads = 10)]
/// # async fn main() {}
/// ```
///
/// The `worker_threads` option configures the number of worker threads, and
/// defaults to the number of cpus on the system. This is the default flavor.
///
/// # Current thread runtime
///
/// To use the single-threaded runtime known as the `current_thread` runtime,
/// the macro can be configured using
///
/// ```
/// #[tokio::main(flavor = "current_thread")]
/// # async fn main() {}
/// ```
///
/// ## Function arguments:
///
/// Arguments are allowed for any functions aside from `main` which is special
///
/// ## Usage
///
/// ### Using the multi-thread runtime
///
/// ```rust
/// #[tokio::main]
/// async fn main() {
/// println!("Hello world");
/// }
/// ```
///
/// Equivalent code not using `#[tokio::main]`
///
/// ```rust
/// fn main() {
/// tokio::runtime::Builder::new_multi_thread()
/// .enable_all()
/// .build()
/// .unwrap()
/// .block_on(async {
/// println!("Hello world");
/// })
/// }
/// ```
///
/// ### Using current thread runtime
///
/// The basic scheduler is single-threaded.
///
/// ```rust
/// #[tokio::main(flavor = "current_thread")]
/// async fn main() {
/// println!("Hello world");
/// }
/// ```
///
/// Equivalent code not using `#[tokio::main]`
///
/// ```rust
/// fn main() {
/// tokio::runtime::Builder::new_current_thread()
/// .enable_all()
/// .build()
/// .unwrap()
/// .block_on(async {
/// println!("Hello world");
/// })
/// }
/// ```
///
/// ### Set number of worker threads
///
/// ```rust
/// #[tokio::main(worker_threads = 2)]
/// async fn main() {
/// println!("Hello world");
/// }
/// ```
///
/// Equivalent code not using `#[tokio::main]`
///
/// ```rust
/// fn main() {
/// tokio::runtime::Builder::new_multi_thread()
/// .worker_threads(2)
/// .enable_all()
/// .build()
/// .unwrap()
/// .block_on(async {
/// println!("Hello world");
/// })
/// }
/// ```
///
/// ### NOTE:
///
/// If you rename the tokio crate in your dependencies this macro will not work.
/// If you must rename the 0.3 version of tokio because you're also using the
/// 0.1 version of tokio, you _must_ make the tokio 0.3 crate available as
/// `tokio` in the module where this macro is expanded.
#[proc_macro_attribute]
#[cfg(not(test))] // Work around for rust-lang/rust#62127
pub fn main(args: TokenStream, item: TokenStream) -> TokenStream {
entry::main(args, item, true)
}
/// Marks async function to be executed by selected runtime. This macro helps set up a `Runtime`
/// without requiring the user to use [Runtime](../tokio/runtime/struct.Runtime.html) or
/// [Builder](../tokio/runtime/struct.builder.html) directly.
///
/// ## Function arguments:
///
/// Arguments are allowed for any functions aside from `main` which is special
///
/// ## Usage
///
/// ### Using default
///
/// ```rust
/// #[tokio::main(flavor = "current_thread")]
/// async fn main() {
/// println!("Hello world");
/// }
/// ```
///
/// Equivalent code not using `#[tokio::main]`
///
/// ```rust
/// fn main() {
/// tokio::runtime::Builder::new_current_thread()
/// .enable_all()
/// .build()
/// .unwrap()
/// .block_on(async {
/// println!("Hello world");
/// })
/// }
/// ```
///
/// ### NOTE:
///
/// If you rename the tokio crate in your dependencies this macro
/// will not work. If you must rename the 0.3 version of tokio because
/// you're also using the 0.1 version of tokio, you _must_ make the
/// tokio 0.3 crate available as `tokio` in the module where this
/// macro is expanded.
#[proc_macro_attribute]
#[cfg(not(test))] // Work around for rust-lang/rust#62127
pub fn main_rt(args: TokenStream, item: TokenStream) -> TokenStream {
entry::main(args, item, false)
}
/// Marks async function to be executed by runtime, suitable to test environment
///
/// ## Usage
///
/// ### Multi-thread runtime
///
/// ```no_run
/// #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
/// async fn my_test() {
/// assert!(true);
/// }
/// ```
///
/// ### Using default
///
/// The default test runtime is single-threaded.
///
/// ```no_run
/// #[tokio::test]
/// async fn my_test() {
/// assert!(true);
/// }
/// ```
///
/// ### NOTE:
///
/// If you rename the tokio crate in your dependencies this macro
/// will not work. If you must rename the 0.3 version of tokio because
/// you're also using the 0.1 version of tokio, you _must_ make the
/// tokio 0.3 crate available as `tokio` in the module where this
/// macro is expanded.
#[proc_macro_attribute]
pub fn test(args: TokenStream, item: TokenStream) -> TokenStream {
entry::test(args, item, true)
}
/// Marks async function to be executed by runtime, suitable to test environment
///
/// ## Usage
///
/// ```no_run
/// #[tokio::test]
/// async fn my_test() {
/// assert!(true);
/// }
/// ```
///
/// ### NOTE:
///
/// If you rename the tokio crate in your dependencies this macro
/// will not work. If you must rename the 0.3 version of tokio because
/// you're also using the 0.1 version of tokio, you _must_ make the
/// tokio 0.3 crate available as `tokio` in the module where this
/// macro is expanded.
#[proc_macro_attribute]
pub fn test_rt(args: TokenStream, item: TokenStream) -> TokenStream {
entry::test(args, item, false)
}
/// Always fails with the error message below.
/// ```text
/// The #[tokio::main] macro requires rt or rt-multi-thread.
/// ```
#[proc_macro_attribute]
pub fn main_fail(_args: TokenStream, _item: TokenStream) -> TokenStream {
syn::Error::new(
proc_macro2::Span::call_site(),
"The #[tokio::main] macro requires rt or rt-multi-thread.",
)
.to_compile_error()
.into()
}
/// Always fails with the error message below.
/// ```text
/// The #[tokio::test] macro requires rt or rt-multi-thread.
/// ```
#[proc_macro_attribute]
pub fn test_fail(_args: TokenStream, _item: TokenStream) -> TokenStream {
syn::Error::new(
proc_macro2::Span::call_site(),
"The #[tokio::test] macro requires rt or rt-multi-thread.",
)
.to_compile_error()
.into()
}
/// Implementation detail of the `select!` macro. This macro is **not** intended
/// to be used as part of the public API and is permitted to change.
#[proc_macro]
#[doc(hidden)]
pub fn select_priv_declare_output_enum(input: TokenStream) -> TokenStream {
select::declare_output_enum(input)
}
-43
View File
@@ -1,43 +0,0 @@
use proc_macro::{TokenStream, TokenTree};
use proc_macro2::Span;
use quote::quote;
use syn::Ident;
pub(crate) fn declare_output_enum(input: TokenStream) -> TokenStream {
// passed in is: `(_ _ _)` with one `_` per branch
let branches = match input.into_iter().next() {
Some(TokenTree::Group(group)) => group.stream().into_iter().count(),
_ => panic!("unexpected macro input"),
};
let variants = (0..branches)
.map(|num| Ident::new(&format!("_{}", num), Span::call_site()))
.collect::<Vec<_>>();
// Use a bitfield to track which futures completed
let mask = Ident::new(
if branches <= 8 {
"u8"
} else if branches <= 16 {
"u16"
} else if branches <= 32 {
"u32"
} else if branches <= 64 {
"u64"
} else {
panic!("up to 64 branches supported");
},
Span::call_site(),
);
TokenStream::from(quote! {
pub(super) enum Out<#( #variants ),*> {
#( #variants(#variants), )*
// Include a `Disabled` variant signifying that all select branches
// failed to resolve.
Disabled,
}
pub(super) type Mask = #mask;
})
}
-11
View File
@@ -1,11 +0,0 @@
# 0.3.0 (October 15, 2020)
- Track `tokio` 0.3 release.
# 0.2.1 (April 17, 2020)
- Add `Future` and `Stream` implementations for `task::Spawn<T>`.
# 0.2.0 (November 25, 2019)
- Initial release
-33
View File
@@ -1,33 +0,0 @@
[package]
name = "tokio-test"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - Update CHANGELOG.md.
# - Create "v0.3.x" git tag.
version = "0.3.0"
edition = "2018"
authors = ["Tokio Contributors <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-test/0.3.0/tokio_test"
description = """
Testing utilities for Tokio- and futures-based code
"""
categories = ["asynchronous", "testing"]
[dependencies]
tokio = { version = "0.3.0", path = "../tokio", features = ["rt", "stream", "sync", "time", "test-util"] }
bytes = "0.6.0"
futures-core = "0.3.0"
[dev-dependencies]
tokio = { version = "0.3.0", path = "../tokio", features = ["full"] }
futures-util = "0.3.0"
[package.metadata.docs.rs]
all-features = true
-25
View File
@@ -1,25 +0,0 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
-13
View File
@@ -1,13 +0,0 @@
# tokio-test
Tokio and Futures based testing utilities
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-487
View File
@@ -1,487 +0,0 @@
#![cfg(not(loom))]
//! A mock type implementing [`AsyncRead`] and [`AsyncWrite`].
//!
//!
//! # Overview
//!
//! Provides a type that implements [`AsyncRead`] + [`AsyncWrite`] that can be configured
//! to handle an arbitrary sequence of read and write operations. This is useful
//! for writing unit tests for networking services as using an actual network
//! type is fairly non deterministic.
//!
//! # Usage
//!
//! Attempting to write data that the mock isn't expecting will result in a
//! panic.
//!
//! [`AsyncRead`]: tokio::io::AsyncRead
//! [`AsyncWrite`]: tokio::io::AsyncWrite
use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
use tokio::sync::mpsc;
use tokio::time::{self, Duration, Instant, Sleep};
use futures_core::ready;
use std::collections::VecDeque;
use std::future::Future;
use std::pin::Pin;
use std::sync::Arc;
use std::task::{self, Poll, Waker};
use std::{cmp, io};
/// An I/O object that follows a predefined script.
///
/// This value is created by `Builder` and implements `AsyncRead` + `AsyncWrite`. It
/// follows the scenario described by the builder and panics otherwise.
#[derive(Debug)]
pub struct Mock {
inner: Inner,
}
/// A handle to send additional actions to the related `Mock`.
#[derive(Debug)]
pub struct Handle {
tx: mpsc::UnboundedSender<Action>,
}
/// Builds `Mock` instances.
#[derive(Debug, Clone, Default)]
pub struct Builder {
// Sequence of actions for the Mock to take
actions: VecDeque<Action>,
}
#[derive(Debug, Clone)]
enum Action {
Read(Vec<u8>),
Write(Vec<u8>),
Wait(Duration),
// Wrapped in Arc so that Builder can be cloned and Send.
// Mock is not cloned as does not need to check Rc for ref counts.
ReadError(Option<Arc<io::Error>>),
WriteError(Option<Arc<io::Error>>),
}
#[derive(Debug)]
struct Inner {
actions: VecDeque<Action>,
waiting: Option<Instant>,
sleep: Option<Sleep>,
read_wait: Option<Waker>,
rx: mpsc::UnboundedReceiver<Action>,
}
impl Builder {
/// Return a new, empty `Builder.
pub fn new() -> Self {
Self::default()
}
/// Sequence a `read` operation.
///
/// The next operation in the mock's script will be to expect a `read` call
/// and return `buf`.
pub fn read(&mut self, buf: &[u8]) -> &mut Self {
self.actions.push_back(Action::Read(buf.into()));
self
}
/// Sequence a `read` operation that produces an error.
///
/// The next operation in the mock's script will be to expect a `read` call
/// and return `error`.
pub fn read_error(&mut self, error: io::Error) -> &mut Self {
let error = Some(error.into());
self.actions.push_back(Action::ReadError(error));
self
}
/// Sequence a `write` operation.
///
/// The next operation in the mock's script will be to expect a `write`
/// call.
pub fn write(&mut self, buf: &[u8]) -> &mut Self {
self.actions.push_back(Action::Write(buf.into()));
self
}
/// Sequence a `write` operation that produces an error.
///
/// The next operation in the mock's script will be to expect a `write`
/// call that provides `error`.
pub fn write_error(&mut self, error: io::Error) -> &mut Self {
let error = Some(error.into());
self.actions.push_back(Action::WriteError(error));
self
}
/// Sequence a wait.
///
/// The next operation in the mock's script will be to wait without doing so
/// for `duration` amount of time.
pub fn wait(&mut self, duration: Duration) -> &mut Self {
let duration = cmp::max(duration, Duration::from_millis(1));
self.actions.push_back(Action::Wait(duration));
self
}
/// Build a `Mock` value according to the defined script.
pub fn build(&mut self) -> Mock {
let (mock, _) = self.build_with_handle();
mock
}
/// Build a `Mock` value paired with a handle
pub fn build_with_handle(&mut self) -> (Mock, Handle) {
let (inner, handle) = Inner::new(self.actions.clone());
let mock = Mock { inner };
(mock, handle)
}
}
impl Handle {
/// Sequence a `read` operation.
///
/// The next operation in the mock's script will be to expect a `read` call
/// and return `buf`.
pub fn read(&mut self, buf: &[u8]) -> &mut Self {
self.tx.send(Action::Read(buf.into())).unwrap();
self
}
/// Sequence a `read` operation error.
///
/// The next operation in the mock's script will be to expect a `read` call
/// and return `error`.
pub fn read_error(&mut self, error: io::Error) -> &mut Self {
let error = Some(error.into());
self.tx.send(Action::ReadError(error)).unwrap();
self
}
/// Sequence a `write` operation.
///
/// The next operation in the mock's script will be to expect a `write`
/// call.
pub fn write(&mut self, buf: &[u8]) -> &mut Self {
self.tx.send(Action::Write(buf.into())).unwrap();
self
}
/// Sequence a `write` operation error.
///
/// The next operation in the mock's script will be to expect a `write`
/// call error.
pub fn write_error(&mut self, error: io::Error) -> &mut Self {
let error = Some(error.into());
self.tx.send(Action::WriteError(error)).unwrap();
self
}
}
impl Inner {
fn new(actions: VecDeque<Action>) -> (Inner, Handle) {
let (tx, rx) = mpsc::unbounded_channel();
let inner = Inner {
actions,
sleep: None,
read_wait: None,
rx,
waiting: None,
};
let handle = Handle { tx };
(inner, handle)
}
fn poll_action(&mut self, cx: &mut task::Context<'_>) -> Poll<Option<Action>> {
use futures_core::stream::Stream;
Pin::new(&mut self.rx).poll_next(cx)
}
fn read(&mut self, dst: &mut ReadBuf<'_>) -> io::Result<()> {
match self.action() {
Some(&mut Action::Read(ref mut data)) => {
// Figure out how much to copy
let n = cmp::min(dst.remaining(), data.len());
// Copy the data into the `dst` slice
dst.put_slice(&data[..n]);
// Drain the data from the source
data.drain(..n);
Ok(())
}
Some(&mut Action::ReadError(ref mut err)) => {
// As the
let err = err.take().expect("Should have been removed from actions.");
let err = Arc::try_unwrap(err).expect("There are no other references.");
Err(err)
}
Some(_) => {
// Either waiting or expecting a write
Err(io::ErrorKind::WouldBlock.into())
}
None => Ok(()),
}
}
fn write(&mut self, mut src: &[u8]) -> io::Result<usize> {
let mut ret = 0;
if self.actions.is_empty() {
return Err(io::ErrorKind::BrokenPipe.into());
}
if let Some(&mut Action::Wait(..)) = self.action() {
return Err(io::ErrorKind::WouldBlock.into());
}
if let Some(&mut Action::WriteError(ref mut err)) = self.action() {
let err = err.take().expect("Should have been removed from actions.");
let err = Arc::try_unwrap(err).expect("There are no other references.");
return Err(err);
}
for i in 0..self.actions.len() {
match self.actions[i] {
Action::Write(ref mut expect) => {
let n = cmp::min(src.len(), expect.len());
assert_eq!(&src[..n], &expect[..n]);
// Drop data that was matched
expect.drain(..n);
src = &src[n..];
ret += n;
if src.is_empty() {
return Ok(ret);
}
}
Action::Wait(..) | Action::WriteError(..) => {
break;
}
_ => {}
}
// TODO: remove write
}
Ok(ret)
}
fn remaining_wait(&mut self) -> Option<Duration> {
match self.action() {
Some(&mut Action::Wait(dur)) => Some(dur),
_ => None,
}
}
fn action(&mut self) -> Option<&mut Action> {
loop {
if self.actions.is_empty() {
return None;
}
match self.actions[0] {
Action::Read(ref mut data) => {
if !data.is_empty() {
break;
}
}
Action::Write(ref mut data) => {
if !data.is_empty() {
break;
}
}
Action::Wait(ref mut dur) => {
if let Some(until) = self.waiting {
let now = Instant::now();
if now < until {
break;
}
} else {
self.waiting = Some(Instant::now() + *dur);
break;
}
}
Action::ReadError(ref mut error) | Action::WriteError(ref mut error) => {
if error.is_some() {
break;
}
}
}
let _action = self.actions.pop_front();
}
self.actions.front_mut()
}
}
// ===== impl Inner =====
impl Mock {
fn maybe_wakeup_reader(&mut self) {
match self.inner.action() {
Some(&mut Action::Read(_)) | Some(&mut Action::ReadError(_)) | None => {
if let Some(waker) = self.inner.read_wait.take() {
waker.wake();
}
}
_ => {}
}
}
}
impl AsyncRead for Mock {
fn poll_read(
mut self: Pin<&mut Self>,
cx: &mut task::Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<io::Result<()>> {
loop {
if let Some(ref mut sleep) = self.inner.sleep {
ready!(Pin::new(sleep).poll(cx));
}
// If a sleep is set, it has already fired
self.inner.sleep = None;
// Capture 'filled' to monitor if it changed
let filled = buf.filled().len();
match self.inner.read(buf) {
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
if let Some(rem) = self.inner.remaining_wait() {
let until = Instant::now() + rem;
self.inner.sleep = Some(time::sleep_until(until));
} else {
self.inner.read_wait = Some(cx.waker().clone());
return Poll::Pending;
}
}
Ok(()) => {
if buf.filled().len() == filled {
match ready!(self.inner.poll_action(cx)) {
Some(action) => {
self.inner.actions.push_back(action);
continue;
}
None => {
return Poll::Ready(Ok(()));
}
}
} else {
return Poll::Ready(Ok(()));
}
}
Err(e) => return Poll::Ready(Err(e)),
}
}
}
}
impl AsyncWrite for Mock {
fn poll_write(
mut self: Pin<&mut Self>,
cx: &mut task::Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
loop {
if let Some(ref mut sleep) = self.inner.sleep {
ready!(Pin::new(sleep).poll(cx));
}
// If a sleep is set, it has already fired
self.inner.sleep = None;
match self.inner.write(buf) {
Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => {
if let Some(rem) = self.inner.remaining_wait() {
let until = Instant::now() + rem;
self.inner.sleep = Some(time::sleep_until(until));
} else {
panic!("unexpected WouldBlock");
}
}
Ok(0) => {
// TODO: Is this correct?
if !self.inner.actions.is_empty() {
return Poll::Pending;
}
// TODO: Extract
match ready!(self.inner.poll_action(cx)) {
Some(action) => {
self.inner.actions.push_back(action);
continue;
}
None => {
panic!("unexpected write");
}
}
}
ret => {
self.maybe_wakeup_reader();
return Poll::Ready(ret);
}
}
}
}
fn poll_flush(self: Pin<&mut Self>, _cx: &mut task::Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut task::Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
}
/// Ensures that Mock isn't dropped with data "inside".
impl Drop for Mock {
fn drop(&mut self) {
// Avoid double panicking, since makes debugging much harder.
if std::thread::panicking() {
return;
}
self.inner.actions.iter().for_each(|a| match a {
Action::Read(data) => assert!(data.is_empty(), "There is still data left to read."),
Action::Write(data) => assert!(data.is_empty(), "There is still data left to write."),
_ => (),
})
}
}
/*
/// Returns `true` if called from the context of a futures-rs Task
fn is_task_ctx() -> bool {
use std::panic;
// Save the existing panic hook
let h = panic::take_hook();
// Install a new one that does nothing
panic::set_hook(Box::new(|_| {}));
// Attempt to call the fn
let r = panic::catch_unwind(|| task::current()).is_ok();
// Re-install the old one
panic::set_hook(h);
// Return the result
r
}
*/
-37
View File
@@ -1,37 +0,0 @@
#![doc(html_root_url = "https://docs.rs/tokio-test/0.3.0")]
#![warn(
missing_debug_implementations,
missing_docs,
rust_2018_idioms,
unreachable_pub
)]
#![cfg_attr(docsrs, deny(broken_intra_doc_links))]
#![doc(test(
no_crate_inject,
attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))
))]
//! Tokio and Futures based testing utilites
pub mod io;
mod macros;
pub mod task;
/// Runs the provided future, blocking the current thread until the
/// future completes.
///
/// For more information, see the documentation for
/// [`tokio::runtime::current_thread::Runtime::block_on`][runtime-block-on].
///
/// [runtime-block-on]: https://docs.rs/tokio/0.2.0-alpha.2/tokio/runtime/current_thread/struct.Runtime.html#method.block_on
pub fn block_on<F: std::future::Future>(future: F) -> F::Output {
use tokio::runtime;
let rt = runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
rt.block_on(future)
}
-261
View File
@@ -1,261 +0,0 @@
//! A collection of useful macros for testing futures and tokio based code
/// Asserts a `Poll` is ready, returning the value.
///
/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready` at
/// runtime.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use futures_util::future;
/// use tokio_test::{assert_ready, task};
///
/// let mut fut = task::spawn(future::ready(()));
/// assert_ready!(fut.poll());
/// ```
#[macro_export]
macro_rules! assert_ready {
($e:expr) => {{
use core::task::Poll::*;
match $e {
Ready(v) => v,
Pending => panic!("pending"),
}
}};
($e:expr, $($msg:tt)+) => {{
use core::task::Poll::*;
match $e {
Ready(v) => v,
Pending => {
panic!("pending; {}", format_args!($($msg)+))
}
}
}};
}
/// Asserts a `Poll<Result<...>>` is ready and `Ok`, returning the value.
///
/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready(Ok(..))` at
/// runtime.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use futures_util::future;
/// use tokio_test::{assert_ready_ok, task};
///
/// let mut fut = task::spawn(future::ok::<_, ()>(()));
/// assert_ready_ok!(fut.poll());
/// ```
#[macro_export]
macro_rules! assert_ready_ok {
($e:expr) => {{
use tokio_test::{assert_ready, assert_ok};
let val = assert_ready!($e);
assert_ok!(val)
}};
($e:expr, $($msg:tt)+) => {{
use tokio_test::{assert_ready, assert_ok};
let val = assert_ready!($e, $($msg)*);
assert_ok!(val, $($msg)*)
}};
}
/// Asserts a `Poll<Result<...>>` is ready and `Err`, returning the error.
///
/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready(Err(..))` at
/// runtime.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use futures_util::future;
/// use tokio_test::{assert_ready_err, task};
///
/// let mut fut = task::spawn(future::err::<(), _>(()));
/// assert_ready_err!(fut.poll());
/// ```
#[macro_export]
macro_rules! assert_ready_err {
($e:expr) => {{
use tokio_test::{assert_ready, assert_err};
let val = assert_ready!($e);
assert_err!(val)
}};
($e:expr, $($msg:tt)+) => {{
use tokio_test::{assert_ready, assert_err};
let val = assert_ready!($e, $($msg)*);
assert_err!(val, $($msg)*)
}};
}
/// Asserts a `Poll` is pending.
///
/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Pending` at
/// runtime.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use futures_util::future;
/// use tokio_test::{assert_pending, task};
///
/// let mut fut = task::spawn(future::pending::<()>());
/// assert_pending!(fut.poll());
/// ```
#[macro_export]
macro_rules! assert_pending {
($e:expr) => {{
use core::task::Poll::*;
match $e {
Pending => {}
Ready(v) => panic!("ready; value = {:?}", v),
}
}};
($e:expr, $($msg:tt)+) => {{
use core::task::Poll::*;
match $e {
Pending => {}
Ready(v) => {
panic!("ready; value = {:?}; {}", v, format_args!($($msg)+))
}
}
}};
}
/// Asserts if a poll is ready and check for equality on the value
///
/// This will invoke `panic!` if the provided `Poll` does not evaluate to `Poll::Ready` at
/// runtime and the value produced does not partially equal the expected value.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use futures_util::future;
/// use tokio_test::{assert_ready_eq, task};
///
/// let mut fut = task::spawn(future::ready(42));
/// assert_ready_eq!(fut.poll(), 42);
/// ```
#[macro_export]
macro_rules! assert_ready_eq {
($e:expr, $expect:expr) => {
let val = $crate::assert_ready!($e);
assert_eq!(val, $expect)
};
($e:expr, $expect:expr, $($msg:tt)+) => {
let val = $crate::assert_ready!($e, $($msg)*);
assert_eq!(val, $expect, $($msg)*)
};
}
/// Asserts that the expression evaluates to `Ok` and returns the value.
///
/// This will invoke the `panic!` macro if the provided expression does not evaluate to `Ok` at
/// runtime.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use tokio_test::assert_ok;
///
/// let n: u32 = assert_ok!("123".parse());
///
/// let s = "123";
/// let n: u32 = assert_ok!(s.parse(), "testing parsing {:?} as a u32", s);
/// ```
#[macro_export]
macro_rules! assert_ok {
($e:expr) => {
assert_ok!($e,)
};
($e:expr,) => {{
use std::result::Result::*;
match $e {
Ok(v) => v,
Err(e) => panic!("assertion failed: Err({:?})", e),
}
}};
($e:expr, $($arg:tt)+) => {{
use std::result::Result::*;
match $e {
Ok(v) => v,
Err(e) => panic!("assertion failed: Err({:?}): {}", e, format_args!($($arg)+)),
}
}};
}
/// Asserts that the expression evaluates to `Err` and returns the error.
///
/// This will invoke the `panic!` macro if the provided expression does not evaluate to `Err` at
/// runtime.
///
/// # Custom Messages
///
/// This macro has a second form, where a custom panic message can be provided with or without
/// arguments for formatting.
///
/// # Examples
///
/// ```
/// use tokio_test::assert_err;
/// use std::str::FromStr;
///
///
/// let err = assert_err!(u32::from_str("fail"));
///
/// let msg = "fail";
/// let err = assert_err!(u32::from_str(msg), "testing parsing {:?} as u32", msg);
/// ```
#[macro_export]
macro_rules! assert_err {
($e:expr) => {
assert_err!($e,);
};
($e:expr,) => {{
use std::result::Result::*;
match $e {
Ok(v) => panic!("assertion failed: Ok({:?})", v),
Err(e) => e,
}
}};
($e:expr, $($arg:tt)+) => {{
use std::result::Result::*;
match $e {
Ok(v) => panic!("assertion failed: Ok({:?}): {}", v, format_args!($($arg)+)),
Err(e) => e,
}
}};
}
-253
View File
@@ -1,253 +0,0 @@
//! Futures task based helpers
#![allow(clippy::mutex_atomic)]
use std::future::Future;
use std::mem;
use std::ops;
use std::pin::Pin;
use std::sync::{Arc, Condvar, Mutex};
use std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker};
use tokio::stream::Stream;
/// TODO: dox
pub fn spawn<T>(task: T) -> Spawn<T> {
Spawn {
task: MockTask::new(),
future: Box::pin(task),
}
}
/// Future spawned on a mock task
#[derive(Debug)]
pub struct Spawn<T> {
task: MockTask,
future: Pin<Box<T>>,
}
/// Mock task
///
/// A mock task is able to intercept and track wake notifications.
#[derive(Debug, Clone)]
struct MockTask {
waker: Arc<ThreadWaker>,
}
#[derive(Debug)]
struct ThreadWaker {
state: Mutex<usize>,
condvar: Condvar,
}
const IDLE: usize = 0;
const WAKE: usize = 1;
const SLEEP: usize = 2;
impl<T> Spawn<T> {
/// Consumes `self` returning the inner value
pub fn into_inner(self) -> T
where
T: Unpin,
{
*Pin::into_inner(self.future)
}
/// Returns `true` if the inner future has received a wake notification
/// since the last call to `enter`.
pub fn is_woken(&self) -> bool {
self.task.is_woken()
}
/// Returns the number of references to the task waker
///
/// The task itself holds a reference. The return value will never be zero.
pub fn waker_ref_count(&self) -> usize {
self.task.waker_ref_count()
}
/// Enter the task context
pub fn enter<F, R>(&mut self, f: F) -> R
where
F: FnOnce(&mut Context<'_>, Pin<&mut T>) -> R,
{
let fut = self.future.as_mut();
self.task.enter(|cx| f(cx, fut))
}
}
impl<T: Unpin> ops::Deref for Spawn<T> {
type Target = T;
fn deref(&self) -> &T {
&self.future
}
}
impl<T: Unpin> ops::DerefMut for Spawn<T> {
fn deref_mut(&mut self) -> &mut T {
&mut self.future
}
}
impl<T: Future> Spawn<T> {
/// Polls a future
pub fn poll(&mut self) -> Poll<T::Output> {
let fut = self.future.as_mut();
self.task.enter(|cx| fut.poll(cx))
}
}
impl<T: Stream> Spawn<T> {
/// Polls a stream
pub fn poll_next(&mut self) -> Poll<Option<T::Item>> {
let stream = self.future.as_mut();
self.task.enter(|cx| stream.poll_next(cx))
}
}
impl<T: Future> Future for Spawn<T> {
type Output = T::Output;
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
self.future.as_mut().poll(cx)
}
}
impl<T: Stream> Stream for Spawn<T> {
type Item = T::Item;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.future.as_mut().poll_next(cx)
}
}
impl MockTask {
/// Creates new mock task
fn new() -> Self {
MockTask {
waker: Arc::new(ThreadWaker::new()),
}
}
/// Runs a closure from the context of the task.
///
/// Any wake notifications resulting from the execution of the closure are
/// tracked.
fn enter<F, R>(&mut self, f: F) -> R
where
F: FnOnce(&mut Context<'_>) -> R,
{
self.waker.clear();
let waker = self.waker();
let mut cx = Context::from_waker(&waker);
f(&mut cx)
}
/// Returns `true` if the inner future has received a wake notification
/// since the last call to `enter`.
fn is_woken(&self) -> bool {
self.waker.is_woken()
}
/// Returns the number of references to the task waker
///
/// The task itself holds a reference. The return value will never be zero.
fn waker_ref_count(&self) -> usize {
Arc::strong_count(&self.waker)
}
fn waker(&self) -> Waker {
unsafe {
let raw = to_raw(self.waker.clone());
Waker::from_raw(raw)
}
}
}
impl Default for MockTask {
fn default() -> Self {
Self::new()
}
}
impl ThreadWaker {
fn new() -> Self {
ThreadWaker {
state: Mutex::new(IDLE),
condvar: Condvar::new(),
}
}
/// Clears any previously received wakes, avoiding potential spurrious
/// wake notifications. This should only be called immediately before running the
/// task.
fn clear(&self) {
*self.state.lock().unwrap() = IDLE;
}
fn is_woken(&self) -> bool {
match *self.state.lock().unwrap() {
IDLE => false,
WAKE => true,
_ => unreachable!(),
}
}
fn wake(&self) {
// First, try transitioning from IDLE -> NOTIFY, this does not require a lock.
let mut state = self.state.lock().unwrap();
let prev = *state;
if prev == WAKE {
return;
}
*state = WAKE;
if prev == IDLE {
return;
}
// The other half is sleeping, so we wake it up.
assert_eq!(prev, SLEEP);
self.condvar.notify_one();
}
}
static VTABLE: RawWakerVTable = RawWakerVTable::new(clone, wake, wake_by_ref, drop_waker);
unsafe fn to_raw(waker: Arc<ThreadWaker>) -> RawWaker {
RawWaker::new(Arc::into_raw(waker) as *const (), &VTABLE)
}
unsafe fn from_raw(raw: *const ()) -> Arc<ThreadWaker> {
Arc::from_raw(raw as *const ThreadWaker)
}
unsafe fn clone(raw: *const ()) -> RawWaker {
let waker = from_raw(raw);
// Increment the ref count
mem::forget(waker.clone());
to_raw(waker)
}
unsafe fn wake(raw: *const ()) {
let waker = from_raw(raw);
waker.wake();
}
unsafe fn wake_by_ref(raw: *const ()) {
let waker = from_raw(raw);
waker.wake();
// We don't actually own a reference to the unparker
mem::forget(waker);
}
unsafe fn drop_waker(raw: *const ()) {
let _ = from_raw(raw);
}
-27
View File
@@ -1,27 +0,0 @@
#![warn(rust_2018_idioms)]
use tokio::time::{sleep_until, Duration, Instant};
use tokio_test::block_on;
#[test]
fn async_block() {
assert_eq!(4, block_on(async { 4 }));
}
async fn five() -> u8 {
5
}
#[test]
fn async_fn() {
assert_eq!(5, block_on(five()));
}
#[test]
fn test_sleep() {
let deadline = Instant::now() + Duration::from_millis(100);
block_on(async {
sleep_until(deadline).await;
});
}
-86
View File
@@ -1,86 +0,0 @@
#![warn(rust_2018_idioms)]
use std::io;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio_test::io::Builder;
#[tokio::test]
async fn read() {
let mut mock = Builder::new().read(b"hello ").read(b"world!").build();
let mut buf = [0; 256];
let n = mock.read(&mut buf).await.expect("read 1");
assert_eq!(&buf[..n], b"hello ");
let n = mock.read(&mut buf).await.expect("read 2");
assert_eq!(&buf[..n], b"world!");
}
#[tokio::test]
async fn read_error() {
let error = io::Error::new(io::ErrorKind::Other, "cruel");
let mut mock = Builder::new()
.read(b"hello ")
.read_error(error)
.read(b"world!")
.build();
let mut buf = [0; 256];
let n = mock.read(&mut buf).await.expect("read 1");
assert_eq!(&buf[..n], b"hello ");
match mock.read(&mut buf).await {
Err(error) => {
assert_eq!(error.kind(), io::ErrorKind::Other);
assert_eq!("cruel", format!("{}", error));
}
Ok(_) => panic!("error not received"),
}
let n = mock.read(&mut buf).await.expect("read 1");
assert_eq!(&buf[..n], b"world!");
}
#[tokio::test]
async fn write() {
let mut mock = Builder::new().write(b"hello ").write(b"world!").build();
mock.write_all(b"hello ").await.expect("write 1");
mock.write_all(b"world!").await.expect("write 2");
}
#[tokio::test]
async fn write_error() {
let error = io::Error::new(io::ErrorKind::Other, "cruel");
let mut mock = Builder::new()
.write(b"hello ")
.write_error(error)
.write(b"world!")
.build();
mock.write_all(b"hello ").await.expect("write 1");
match mock.write_all(b"whoa").await {
Err(error) => {
assert_eq!(error.kind(), io::ErrorKind::Other);
assert_eq!("cruel", format!("{}", error));
}
Ok(_) => panic!("error not received"),
}
mock.write_all(b"world!").await.expect("write 2");
}
#[tokio::test]
#[should_panic]
async fn mock_panics_read_data_left() {
use tokio_test::io::Builder;
Builder::new().read(b"read").build();
}
#[tokio::test]
#[should_panic]
async fn mock_panics_write_data_left() {
use tokio_test::io::Builder;
Builder::new().write(b"write").build();
}
-107
View File
@@ -1,107 +0,0 @@
#![warn(rust_2018_idioms)]
use std::task::Poll;
use tokio_test::{
assert_pending, assert_ready, assert_ready_eq, assert_ready_err, assert_ready_ok,
};
fn ready() -> Poll<()> {
Poll::Ready(())
}
fn ready_ok() -> Poll<Result<(), ()>> {
Poll::Ready(Ok(()))
}
fn ready_err() -> Poll<Result<(), ()>> {
Poll::Ready(Err(()))
}
fn pending() -> Poll<()> {
Poll::Pending
}
#[derive(Debug)]
enum Test {
Data,
}
#[test]
fn assert_ready() {
let poll = ready();
assert_ready!(poll);
assert_ready!(poll, "some message");
assert_ready!(poll, "{:?}", ());
assert_ready!(poll, "{:?}", Test::Data);
}
#[test]
#[should_panic]
fn assert_ready_on_pending() {
let poll = pending();
assert_ready!(poll);
}
#[test]
fn assert_pending() {
let poll = pending();
assert_pending!(poll);
assert_pending!(poll, "some message");
assert_pending!(poll, "{:?}", ());
assert_pending!(poll, "{:?}", Test::Data);
}
#[test]
#[should_panic]
fn assert_pending_on_ready() {
let poll = ready();
assert_pending!(poll);
}
#[test]
fn assert_ready_ok() {
let poll = ready_ok();
assert_ready_ok!(poll);
assert_ready_ok!(poll, "some message");
assert_ready_ok!(poll, "{:?}", ());
assert_ready_ok!(poll, "{:?}", Test::Data);
}
#[test]
#[should_panic]
fn assert_ok_on_err() {
let poll = ready_err();
assert_ready_ok!(poll);
}
#[test]
fn assert_ready_err() {
let poll = ready_err();
assert_ready_err!(poll);
assert_ready_err!(poll, "some message");
assert_ready_err!(poll, "{:?}", ());
assert_ready_err!(poll, "{:?}", Test::Data);
}
#[test]
#[should_panic]
fn assert_err_on_ok() {
let poll = ready_ok();
assert_ready_err!(poll);
}
#[test]
fn assert_ready_eq() {
let poll = ready();
assert_ready_eq!(poll, ());
assert_ready_eq!(poll, (), "some message");
assert_ready_eq!(poll, (), "{:?}", ());
assert_ready_eq!(poll, (), "{:?}", Test::Data);
}
#[test]
#[should_panic]
fn assert_eq_on_not_eq() {
let poll = ready_err();
assert_ready_eq!(poll, Ok(()));
}
-52
View File
@@ -1,52 +0,0 @@
# 0.5.1 (December 3, 2020)
### Added
- io: `poll_read_buf` util fn (#2972).
- io: `poll_write_buf` util fn with vectored write support (#3156).
# 0.5.0 (October 30, 2020)
### Changed
- io: update `bytes` to 0.6 (#3071).
# 0.4.0 (October 15, 2020)
### Added
- sync: `CancellationToken` for coordinating task cancellation (#2747).
- rt: `TokioContext` sets the Tokio runtime for the duration of a future (#2791)
- io: `StreamReader`/`ReaderStream` map between `AsyncRead` values and `Stream`
of bytes (#2788).
- time: `DelayQueue` to manage many delays (#2897).
# 0.3.1 (March 18, 2020)
### Fixed
- Adjust minimum-supported Tokio version to v0.2.5 to account for an internal
dependency on features in that version of Tokio. ([#2326])
# 0.3.0 (March 4, 2020)
### Changed
- **Breaking Change**: Change `Encoder` trait to take a generic `Item` parameter, which allows
codec writers to pass references into `Framed` and `FramedWrite` types. ([#1746])
### Added
- Add futures-io/tokio::io compatibility layer. ([#2117])
- Add `Framed::with_capacity`. ([#2215])
### Fixed
- Use advance over split_to when data is not needed. ([#2198])
# 0.2.0 (November 26, 2019)
- Initial release
[#2326]: https://github.com/tokio-rs/tokio/pull/2326
[#2215]: https://github.com/tokio-rs/tokio/pull/2215
[#2198]: https://github.com/tokio-rs/tokio/pull/2198
[#2117]: https://github.com/tokio-rs/tokio/pull/2117
[#1746]: https://github.com/tokio-rs/tokio/pull/1746
-56
View File
@@ -1,56 +0,0 @@
[package]
name = "tokio-util"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - Update CHANGELOG.md.
# - Create "v0.2.x" git tag.
version = "0.5.1"
edition = "2018"
authors = ["Tokio Contributors <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-util/0.5.1/tokio_util"
description = """
Additional utilities for working with Tokio.
"""
categories = ["asynchronous"]
[features]
# No features on by default
default = []
# Shorthand for enabling everything
full = ["codec", "compat", "io", "time", "net", "rt"]
net = ["tokio/net"]
compat = ["futures-io",]
codec = ["tokio/stream"]
time = ["tokio/time","slab"]
io = []
rt = ["tokio/rt"]
[dependencies]
tokio = { version = "0.3.4", path = "../tokio" }
bytes = "0.6.0"
futures-core = "0.3.0"
futures-sink = "0.3.0"
futures-io = { version = "0.3.0", optional = true }
log = "0.4"
pin-project-lite = "0.2.0"
slab = { version = "0.4.1", optional = true } # Backs `DelayQueue`
[dev-dependencies]
tokio = { version = "0.3.0", path = "../tokio", features = ["full"] }
tokio-test = { version = "0.3.0", path = "../tokio-test" }
futures = "0.3.0"
futures-test = "0.3.5"
[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "docsrs"]
-25
View File
@@ -1,25 +0,0 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
-13
View File
@@ -1,13 +0,0 @@
# tokio-util
Utilities for encoding and decoding frames.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-49
View File
@@ -1,49 +0,0 @@
macro_rules! cfg_codec {
($($item:item)*) => {
$(
#[cfg(feature = "codec")]
#[cfg_attr(docsrs, doc(cfg(feature = "codec")))]
$item
)*
}
}
macro_rules! cfg_compat {
($($item:item)*) => {
$(
#[cfg(feature = "compat")]
#[cfg_attr(docsrs, doc(cfg(feature = "compat")))]
$item
)*
}
}
macro_rules! cfg_net {
($($item:item)*) => {
$(
#[cfg(all(feature = "net", feature = "codec"))]
#[cfg_attr(docsrs, doc(cfg(all(feature = "net", feature = "codec"))))]
$item
)*
}
}
macro_rules! cfg_io {
($($item:item)*) => {
$(
#[cfg(feature = "io")]
#[cfg_attr(docsrs, doc(cfg(feature = "io")))]
$item
)*
}
}
macro_rules! cfg_rt {
($($item:item)*) => {
$(
#[cfg(feature = "rt")]
#[cfg_attr(docsrs, doc(cfg(feature = "rt")))]
$item
)*
}
}
-76
View File
@@ -1,76 +0,0 @@
use crate::codec::decoder::Decoder;
use crate::codec::encoder::Encoder;
use bytes::{BufMut, Bytes, BytesMut};
use std::io;
/// A simple [`Decoder`] and [`Encoder`] implementation that just ships bytes around.
///
/// [`Decoder`]: crate::codec::Decoder
/// [`Encoder`]: crate::codec::Encoder
///
/// # Example
///
/// Turn an [`AsyncRead`] into a stream of `Result<`[`BytesMut`]`, `[`Error`]`>`.
///
/// [`AsyncRead`]: tokio::io::AsyncRead
/// [`BytesMut`]: bytes::BytesMut
/// [`Error`]: std::io::Error
///
/// ```
/// # mod hidden {
/// # #[allow(unused_imports)]
/// use tokio::fs::File;
/// # }
/// use tokio::io::AsyncRead;
/// use tokio_util::codec::{FramedRead, BytesCodec};
///
/// # enum File {}
/// # impl File {
/// # async fn open(_name: &str) -> Result<impl AsyncRead, std::io::Error> {
/// # use std::io::Cursor;
/// # Ok(Cursor::new(vec![0, 1, 2, 3, 4, 5]))
/// # }
/// # }
/// #
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() -> Result<(), std::io::Error> {
/// let my_async_read = File::open("filename.txt").await?;
/// let my_stream_of_bytes = FramedRead::new(my_async_read, BytesCodec::new());
/// # Ok(())
/// # }
/// ```
///
#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash, Default)]
pub struct BytesCodec(());
impl BytesCodec {
/// Creates a new `BytesCodec` for shipping around raw bytes.
pub fn new() -> BytesCodec {
BytesCodec(())
}
}
impl Decoder for BytesCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<BytesMut>, io::Error> {
if !buf.is_empty() {
let len = buf.len();
Ok(Some(buf.split_to(len)))
} else {
Ok(None)
}
}
}
impl Encoder<Bytes> for BytesCodec {
type Error = io::Error;
fn encode(&mut self, data: Bytes, buf: &mut BytesMut) -> Result<(), io::Error> {
buf.reserve(data.len());
buf.put(data);
Ok(())
}
}
-165
View File
@@ -1,165 +0,0 @@
use crate::codec::Framed;
use tokio::io::{AsyncRead, AsyncWrite};
use bytes::BytesMut;
use std::io;
/// Decoding of frames via buffers.
///
/// This trait is used when constructing an instance of [`Framed`] or
/// [`FramedRead`]. An implementation of `Decoder` takes a byte stream that has
/// already been buffered in `src` and decodes the data into a stream of
/// `Self::Item` frames.
///
/// Implementations are able to track state on `self`, which enables
/// implementing stateful streaming parsers. In many cases, though, this type
/// will simply be a unit struct (e.g. `struct HttpDecoder`).
///
/// [`Framed`]: crate::codec::Framed
/// [`FramedRead`]: crate::codec::FramedRead
pub trait Decoder {
/// The type of decoded frames.
type Item;
/// The type of unrecoverable frame decoding errors.
///
/// If an individual message is ill-formed but can be ignored without
/// interfering with the processing of future messages, it may be more
/// useful to report the failure as an `Item`.
///
/// `From<io::Error>` is required in the interest of making `Error` suitable
/// for returning directly from a [`FramedRead`], and to enable the default
/// implementation of `decode_eof` to yield an `io::Error` when the decoder
/// fails to consume all available data.
///
/// Note that implementors of this trait can simply indicate `type Error =
/// io::Error` to use I/O errors as this type.
///
/// [`FramedRead`]: crate::codec::FramedRead
type Error: From<io::Error>;
/// Attempts to decode a frame from the provided buffer of bytes.
///
/// This method is called by [`FramedRead`] whenever bytes are ready to be
/// parsed. The provided buffer of bytes is what's been read so far, and
/// this instance of `Decode` can determine whether an entire frame is in
/// the buffer and is ready to be returned.
///
/// If an entire frame is available, then this instance will remove those
/// bytes from the buffer provided and return them as a decoded
/// frame. Note that removing bytes from the provided buffer doesn't always
/// necessarily copy the bytes, so this should be an efficient operation in
/// most circumstances.
///
/// If the bytes look valid, but a frame isn't fully available yet, then
/// `Ok(None)` is returned. This indicates to the [`Framed`] instance that
/// it needs to read some more bytes before calling this method again.
///
/// Note that the bytes provided may be empty. If a previous call to
/// `decode` consumed all the bytes in the buffer then `decode` will be
/// called again until it returns `Ok(None)`, indicating that more bytes need to
/// be read.
///
/// Finally, if the bytes in the buffer are malformed then an error is
/// returned indicating why. This informs [`Framed`] that the stream is now
/// corrupt and should be terminated.
///
/// [`Framed`]: crate::codec::Framed
/// [`FramedRead`]: crate::codec::FramedRead
///
/// # Buffer management
///
/// Before returning from the function, implementations should ensure that
/// the buffer has appropriate capacity in anticipation of future calls to
/// `decode`. Failing to do so leads to inefficiency.
///
/// For example, if frames have a fixed length, or if the length of the
/// current frame is known from a header, a possible buffer management
/// strategy is:
///
/// ```no_run
/// # use std::io;
/// #
/// # use bytes::BytesMut;
/// # use tokio_util::codec::Decoder;
/// #
/// # struct MyCodec;
/// #
/// impl Decoder for MyCodec {
/// // ...
/// # type Item = BytesMut;
/// # type Error = io::Error;
///
/// fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
/// // ...
///
/// // Reserve enough to complete decoding of the current frame.
/// let current_frame_len: usize = 1000; // Example.
/// // And to start decoding the next frame.
/// let next_frame_header_len: usize = 10; // Example.
/// src.reserve(current_frame_len + next_frame_header_len);
///
/// return Ok(None);
/// }
/// }
/// ```
///
/// An optimal buffer management strategy minimizes reallocations and
/// over-allocations.
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error>;
/// A default method available to be called when there are no more bytes
/// available to be read from the underlying I/O.
///
/// This method defaults to calling `decode` and returns an error if
/// `Ok(None)` is returned while there is unconsumed data in `buf`.
/// Typically this doesn't need to be implemented unless the framing
/// protocol differs near the end of the stream.
///
/// Note that the `buf` argument may be empty. If a previous call to
/// `decode_eof` consumed all the bytes in the buffer, `decode_eof` will be
/// called again until it returns `None`, indicating that there are no more
/// frames to yield. This behavior enables returning finalization frames
/// that may not be based on inbound data.
fn decode_eof(&mut self, buf: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
match self.decode(buf)? {
Some(frame) => Ok(Some(frame)),
None => {
if buf.is_empty() {
Ok(None)
} else {
Err(io::Error::new(io::ErrorKind::Other, "bytes remaining on stream").into())
}
}
}
}
/// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this
/// `Io` object, using `Decode` and `Encode` to read and write the raw data.
///
/// Raw I/O objects work with byte sequences, but higher-level code usually
/// wants to batch these into meaningful chunks, called "frames". This
/// method layers framing on top of an I/O object, by using the `Codec`
/// traits to handle encoding and decoding of messages frames. Note that
/// the incoming and outgoing frame types may be distinct.
///
/// This function returns a *single* object that is both `Stream` and
/// `Sink`; grouping this into a single object is often useful for layering
/// things like gzip or TLS, which require both read and write access to the
/// underlying object.
///
/// If you want to work more directly with the streams and sink, consider
/// calling `split` on the [`Framed`] returned by this method, which will
/// break them into separate objects, allowing them to interact more easily.
///
/// [`Stream`]: tokio::stream::Stream
/// [`Sink`]: futures_sink::Sink
/// [`Framed`]: crate::codec::Framed
fn framed<T: AsyncRead + AsyncWrite + Sized>(self, io: T) -> Framed<T, Self>
where
Self: Sized,
{
Framed::new(io, self)
}
}
-25
View File
@@ -1,25 +0,0 @@
use bytes::BytesMut;
use std::io;
/// Trait of helper objects to write out messages as bytes, for use with
/// [`FramedWrite`].
///
/// [`FramedWrite`]: crate::codec::FramedWrite
pub trait Encoder<Item> {
/// The type of encoding errors.
///
/// [`FramedWrite`] requires `Encoder`s errors to implement `From<io::Error>`
/// in the interest letting it return `Error`s directly.
///
/// [`FramedWrite`]: crate::codec::FramedWrite
type Error: From<io::Error>;
/// Encodes a frame into the buffer provided.
///
/// This method will encode `item` into the byte buffer provided by `dst`.
/// The `dst` provided is an internal buffer of the [`FramedWrite`] instance and
/// will be written out when possible.
///
/// [`FramedWrite`]: crate::codec::FramedWrite
fn encode(&mut self, item: Item, dst: &mut BytesMut) -> Result<(), Self::Error>;
}
-320
View File
@@ -1,320 +0,0 @@
use crate::codec::decoder::Decoder;
use crate::codec::encoder::Encoder;
use crate::codec::framed_impl::{FramedImpl, RWFrames, ReadFrame, WriteFrame};
use tokio::{
io::{AsyncRead, AsyncWrite},
stream::Stream,
};
use bytes::BytesMut;
use futures_sink::Sink;
use pin_project_lite::pin_project;
use std::fmt;
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
pin_project! {
/// A unified [`Stream`] and [`Sink`] interface to an underlying I/O object, using
/// the `Encoder` and `Decoder` traits to encode and decode frames.
///
/// You can create a `Framed` instance by using the [`Decoder::framed`] adapter, or
/// by using the `new` function seen below.
///
/// [`Stream`]: tokio::stream::Stream
/// [`Sink`]: futures_sink::Sink
/// [`AsyncRead`]: tokio::io::AsyncRead
/// [`Decoder::framed`]: crate::codec::Decoder::framed()
pub struct Framed<T, U> {
#[pin]
inner: FramedImpl<T, U, RWFrames>
}
}
impl<T, U> Framed<T, U>
where
T: AsyncRead + AsyncWrite,
{
/// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this
/// I/O object, using [`Decoder`] and [`Encoder`] to read and write the raw data.
///
/// Raw I/O objects work with byte sequences, but higher-level code usually
/// wants to batch these into meaningful chunks, called "frames". This
/// method layers framing on top of an I/O object, by using the codec
/// traits to handle encoding and decoding of messages frames. Note that
/// the incoming and outgoing frame types may be distinct.
///
/// This function returns a *single* object that is both [`Stream`] and
/// [`Sink`]; grouping this into a single object is often useful for layering
/// things like gzip or TLS, which require both read and write access to the
/// underlying object.
///
/// If you want to work more directly with the streams and sink, consider
/// calling [`split`] on the `Framed` returned by this method, which will
/// break them into separate objects, allowing them to interact more easily.
///
/// [`Stream`]: tokio::stream::Stream
/// [`Sink`]: futures_sink::Sink
/// [`Decode`]: crate::codec::Decoder
/// [`Encoder`]: crate::codec::Encoder
/// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split
pub fn new(inner: T, codec: U) -> Framed<T, U> {
Framed {
inner: FramedImpl {
inner,
codec,
state: Default::default(),
},
}
}
/// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this
/// I/O object, using [`Decoder`] and [`Encoder`] to read and write the raw data,
/// with a specific read buffer initial capacity.
///
/// Raw I/O objects work with byte sequences, but higher-level code usually
/// wants to batch these into meaningful chunks, called "frames". This
/// method layers framing on top of an I/O object, by using the codec
/// traits to handle encoding and decoding of messages frames. Note that
/// the incoming and outgoing frame types may be distinct.
///
/// This function returns a *single* object that is both [`Stream`] and
/// [`Sink`]; grouping this into a single object is often useful for layering
/// things like gzip or TLS, which require both read and write access to the
/// underlying object.
///
/// If you want to work more directly with the streams and sink, consider
/// calling [`split`] on the `Framed` returned by this method, which will
/// break them into separate objects, allowing them to interact more easily.
///
/// [`Stream`]: tokio::stream::Stream
/// [`Sink`]: futures_sink::Sink
/// [`Decode`]: crate::codec::Decoder
/// [`Encoder`]: crate::codec::Encoder
/// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split
pub fn with_capacity(inner: T, codec: U, capacity: usize) -> Framed<T, U> {
Framed {
inner: FramedImpl {
inner,
codec,
state: RWFrames {
read: ReadFrame {
eof: false,
is_readable: false,
buffer: BytesMut::with_capacity(capacity),
},
write: WriteFrame::default(),
},
},
}
}
}
impl<T, U> Framed<T, U> {
/// Provides a [`Stream`] and [`Sink`] interface for reading and writing to this
/// I/O object, using [`Decoder`] and [`Encoder`] to read and write the raw data.
///
/// Raw I/O objects work with byte sequences, but higher-level code usually
/// wants to batch these into meaningful chunks, called "frames". This
/// method layers framing on top of an I/O object, by using the `Codec`
/// traits to handle encoding and decoding of messages frames. Note that
/// the incoming and outgoing frame types may be distinct.
///
/// This function returns a *single* object that is both [`Stream`] and
/// [`Sink`]; grouping this into a single object is often useful for layering
/// things like gzip or TLS, which require both read and write access to the
/// underlying object.
///
/// This objects takes a stream and a readbuffer and a writebuffer. These field
/// can be obtained from an existing `Framed` with the [`into_parts`] method.
///
/// If you want to work more directly with the streams and sink, consider
/// calling [`split`] on the `Framed` returned by this method, which will
/// break them into separate objects, allowing them to interact more easily.
///
/// [`Stream`]: tokio::stream::Stream
/// [`Sink`]: futures_sink::Sink
/// [`Decoder`]: crate::codec::Decoder
/// [`Encoder`]: crate::codec::Encoder
/// [`into_parts`]: crate::codec::Framed::into_parts()
/// [`split`]: https://docs.rs/futures/0.3/futures/stream/trait.StreamExt.html#method.split
pub fn from_parts(parts: FramedParts<T, U>) -> Framed<T, U> {
Framed {
inner: FramedImpl {
inner: parts.io,
codec: parts.codec,
state: RWFrames {
read: parts.read_buf.into(),
write: parts.write_buf.into(),
},
},
}
}
/// Returns a reference to the underlying I/O stream wrapped by
/// `Framed`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_ref(&self) -> &T {
&self.inner.inner
}
/// Returns a mutable reference to the underlying I/O stream wrapped by
/// `Framed`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner.inner
}
/// Returns a reference to the underlying codec wrapped by
/// `Framed`.
///
/// Note that care should be taken to not tamper with the underlying codec
/// as it may corrupt the stream of frames otherwise being worked with.
pub fn codec(&self) -> &U {
&self.inner.codec
}
/// Returns a mutable reference to the underlying codec wrapped by
/// `Framed`.
///
/// Note that care should be taken to not tamper with the underlying codec
/// as it may corrupt the stream of frames otherwise being worked with.
pub fn codec_mut(&mut self) -> &mut U {
&mut self.inner.codec
}
/// Returns a reference to the read buffer.
pub fn read_buffer(&self) -> &BytesMut {
&self.inner.state.read.buffer
}
/// Returns a mutable reference to the read buffer.
pub fn read_buffer_mut(&mut self) -> &mut BytesMut {
&mut self.inner.state.read.buffer
}
/// Consumes the `Framed`, returning its underlying I/O stream.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_inner(self) -> T {
self.inner.inner
}
/// Consumes the `Framed`, returning its underlying I/O stream, the buffer
/// with unprocessed data, and the codec.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_parts(self) -> FramedParts<T, U> {
FramedParts {
io: self.inner.inner,
codec: self.inner.codec,
read_buf: self.inner.state.read.buffer,
write_buf: self.inner.state.write.buffer,
_priv: (),
}
}
}
// This impl just defers to the underlying FramedImpl
impl<T, U> Stream for Framed<T, U>
where
T: AsyncRead,
U: Decoder,
{
type Item = Result<U::Item, U::Error>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.project().inner.poll_next(cx)
}
}
// This impl just defers to the underlying FramedImpl
impl<T, I, U> Sink<I> for Framed<T, U>
where
T: AsyncWrite,
U: Encoder<I>,
U::Error: From<io::Error>,
{
type Error = U::Error;
fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.poll_ready(cx)
}
fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {
self.project().inner.start_send(item)
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.poll_flush(cx)
}
fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.poll_close(cx)
}
}
impl<T, U> fmt::Debug for Framed<T, U>
where
T: fmt::Debug,
U: fmt::Debug,
{
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("Framed")
.field("io", self.get_ref())
.field("codec", self.codec())
.finish()
}
}
/// `FramedParts` contains an export of the data of a Framed transport.
/// It can be used to construct a new [`Framed`] with a different codec.
/// It contains all current buffers and the inner transport.
///
/// [`Framed`]: crate::codec::Framed
#[derive(Debug)]
#[allow(clippy::manual_non_exhaustive)]
pub struct FramedParts<T, U> {
/// The inner transport used to read bytes to and write bytes to
pub io: T,
/// The codec
pub codec: U,
/// The buffer with read but unprocessed data.
pub read_buf: BytesMut,
/// A buffer with unprocessed data which are not written yet.
pub write_buf: BytesMut,
/// This private field allows us to add additional fields in the future in a
/// backwards compatible way.
_priv: (),
}
impl<T, U> FramedParts<T, U> {
/// Create a new, default, `FramedParts`
pub fn new<I>(io: T, codec: U) -> FramedParts<T, U>
where
U: Encoder<I>,
{
FramedParts {
io,
codec,
read_buf: BytesMut::new(),
write_buf: BytesMut::new(),
_priv: (),
}
}
}
-225
View File
@@ -1,225 +0,0 @@
use crate::codec::decoder::Decoder;
use crate::codec::encoder::Encoder;
use tokio::{
io::{AsyncRead, AsyncWrite},
stream::Stream,
};
use bytes::BytesMut;
use futures_core::ready;
use futures_sink::Sink;
use log::trace;
use pin_project_lite::pin_project;
use std::borrow::{Borrow, BorrowMut};
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
pin_project! {
#[derive(Debug)]
pub(crate) struct FramedImpl<T, U, State> {
#[pin]
pub(crate) inner: T,
pub(crate) state: State,
pub(crate) codec: U,
}
}
const INITIAL_CAPACITY: usize = 8 * 1024;
const BACKPRESSURE_BOUNDARY: usize = INITIAL_CAPACITY;
pub(crate) struct ReadFrame {
pub(crate) eof: bool,
pub(crate) is_readable: bool,
pub(crate) buffer: BytesMut,
}
pub(crate) struct WriteFrame {
pub(crate) buffer: BytesMut,
}
#[derive(Default)]
pub(crate) struct RWFrames {
pub(crate) read: ReadFrame,
pub(crate) write: WriteFrame,
}
impl Default for ReadFrame {
fn default() -> Self {
Self {
eof: false,
is_readable: false,
buffer: BytesMut::with_capacity(INITIAL_CAPACITY),
}
}
}
impl Default for WriteFrame {
fn default() -> Self {
Self {
buffer: BytesMut::with_capacity(INITIAL_CAPACITY),
}
}
}
impl From<BytesMut> for ReadFrame {
fn from(mut buffer: BytesMut) -> Self {
let size = buffer.capacity();
if size < INITIAL_CAPACITY {
buffer.reserve(INITIAL_CAPACITY - size);
}
Self {
buffer,
is_readable: size > 0,
eof: false,
}
}
}
impl From<BytesMut> for WriteFrame {
fn from(mut buffer: BytesMut) -> Self {
let size = buffer.capacity();
if size < INITIAL_CAPACITY {
buffer.reserve(INITIAL_CAPACITY - size);
}
Self { buffer }
}
}
impl Borrow<ReadFrame> for RWFrames {
fn borrow(&self) -> &ReadFrame {
&self.read
}
}
impl BorrowMut<ReadFrame> for RWFrames {
fn borrow_mut(&mut self) -> &mut ReadFrame {
&mut self.read
}
}
impl Borrow<WriteFrame> for RWFrames {
fn borrow(&self) -> &WriteFrame {
&self.write
}
}
impl BorrowMut<WriteFrame> for RWFrames {
fn borrow_mut(&mut self) -> &mut WriteFrame {
&mut self.write
}
}
impl<T, U, R> Stream for FramedImpl<T, U, R>
where
T: AsyncRead,
U: Decoder,
R: BorrowMut<ReadFrame>,
{
type Item = Result<U::Item, U::Error>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
use crate::util::poll_read_buf;
let mut pinned = self.project();
let state: &mut ReadFrame = pinned.state.borrow_mut();
loop {
// Repeatedly call `decode` or `decode_eof` as long as it is
// "readable". Readable is defined as not having returned `None`. If
// the upstream has returned EOF, and the decoder is no longer
// readable, it can be assumed that the decoder will never become
// readable again, at which point the stream is terminated.
if state.is_readable {
if state.eof {
let frame = pinned.codec.decode_eof(&mut state.buffer)?;
return Poll::Ready(frame.map(Ok));
}
trace!("attempting to decode a frame");
if let Some(frame) = pinned.codec.decode(&mut state.buffer)? {
trace!("frame decoded from buffer");
return Poll::Ready(Some(Ok(frame)));
}
state.is_readable = false;
}
assert!(!state.eof);
// Otherwise, try to read more data and try again. Make sure we've
// got room for at least one byte to read to ensure that we don't
// get a spurious 0 that looks like EOF
state.buffer.reserve(1);
let bytect = match poll_read_buf(pinned.inner.as_mut(), cx, &mut state.buffer)? {
Poll::Ready(ct) => ct,
Poll::Pending => return Poll::Pending,
};
if bytect == 0 {
state.eof = true;
}
state.is_readable = true;
}
}
}
impl<T, I, U, W> Sink<I> for FramedImpl<T, U, W>
where
T: AsyncWrite,
U: Encoder<I>,
U::Error: From<io::Error>,
W: BorrowMut<WriteFrame>,
{
type Error = U::Error;
fn poll_ready(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
if self.state.borrow().buffer.len() >= BACKPRESSURE_BOUNDARY {
self.as_mut().poll_flush(cx)
} else {
Poll::Ready(Ok(()))
}
}
fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {
let pinned = self.project();
pinned
.codec
.encode(item, &mut pinned.state.borrow_mut().buffer)?;
Ok(())
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
use crate::util::poll_write_buf;
trace!("flushing framed transport");
let mut pinned = self.project();
while !pinned.state.borrow_mut().buffer.is_empty() {
let WriteFrame { buffer } = pinned.state.borrow_mut();
trace!("writing; remaining={}", buffer.len());
let n = ready!(poll_write_buf(pinned.inner.as_mut(), cx, buffer))?;
if n == 0 {
return Poll::Ready(Err(io::Error::new(
io::ErrorKind::WriteZero,
"failed to \
write frame to transport",
)
.into()));
}
}
// Try flushing the underlying IO
ready!(pinned.inner.poll_flush(cx))?;
trace!("framed transport flushed");
Poll::Ready(Ok(()))
}
fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
ready!(self.as_mut().poll_flush(cx))?;
ready!(self.project().inner.poll_shutdown(cx))?;
Poll::Ready(Ok(()))
}
}
-161
View File
@@ -1,161 +0,0 @@
use crate::codec::framed_impl::{FramedImpl, ReadFrame};
use crate::codec::Decoder;
use tokio::{io::AsyncRead, stream::Stream};
use bytes::BytesMut;
use futures_sink::Sink;
use pin_project_lite::pin_project;
use std::fmt;
use std::pin::Pin;
use std::task::{Context, Poll};
pin_project! {
/// A [`Stream`] of messages decoded from an [`AsyncRead`].
///
/// [`Stream`]: tokio::stream::Stream
/// [`AsyncRead`]: tokio::io::AsyncRead
pub struct FramedRead<T, D> {
#[pin]
inner: FramedImpl<T, D, ReadFrame>,
}
}
// ===== impl FramedRead =====
impl<T, D> FramedRead<T, D>
where
T: AsyncRead,
D: Decoder,
{
/// Creates a new `FramedRead` with the given `decoder`.
pub fn new(inner: T, decoder: D) -> FramedRead<T, D> {
FramedRead {
inner: FramedImpl {
inner,
codec: decoder,
state: Default::default(),
},
}
}
/// Creates a new `FramedRead` with the given `decoder` and a buffer of `capacity`
/// initial size.
pub fn with_capacity(inner: T, decoder: D, capacity: usize) -> FramedRead<T, D> {
FramedRead {
inner: FramedImpl {
inner,
codec: decoder,
state: ReadFrame {
eof: false,
is_readable: false,
buffer: BytesMut::with_capacity(capacity),
},
},
}
}
}
impl<T, D> FramedRead<T, D> {
/// Returns a reference to the underlying I/O stream wrapped by
/// `FramedRead`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_ref(&self) -> &T {
&self.inner.inner
}
/// Returns a mutable reference to the underlying I/O stream wrapped by
/// `FramedRead`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner.inner
}
/// Consumes the `FramedRead`, returning its underlying I/O stream.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_inner(self) -> T {
self.inner.inner
}
/// Returns a reference to the underlying decoder.
pub fn decoder(&self) -> &D {
&self.inner.codec
}
/// Returns a mutable reference to the underlying decoder.
pub fn decoder_mut(&mut self) -> &mut D {
&mut self.inner.codec
}
/// Returns a reference to the read buffer.
pub fn read_buffer(&self) -> &BytesMut {
&self.inner.state.buffer
}
/// Returns a mutable reference to the read buffer.
pub fn read_buffer_mut(&mut self) -> &mut BytesMut {
&mut self.inner.state.buffer
}
}
// This impl just defers to the underlying FramedImpl
impl<T, D> Stream for FramedRead<T, D>
where
T: AsyncRead,
D: Decoder,
{
type Item = Result<D::Item, D::Error>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.project().inner.poll_next(cx)
}
}
// This impl just defers to the underlying T: Sink
impl<T, I, D> Sink<I> for FramedRead<T, D>
where
T: Sink<I>,
{
type Error = T::Error;
fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.project().inner.poll_ready(cx)
}
fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {
self.project().inner.project().inner.start_send(item)
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.project().inner.poll_flush(cx)
}
fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.project().inner.poll_close(cx)
}
}
impl<T, D> fmt::Debug for FramedRead<T, D>
where
T: fmt::Debug,
D: fmt::Debug,
{
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("FramedRead")
.field("inner", &self.get_ref())
.field("decoder", &self.decoder())
.field("eof", &self.inner.state.eof)
.field("is_readable", &self.inner.state.is_readable)
.field("buffer", &self.read_buffer())
.finish()
}
}
-130
View File
@@ -1,130 +0,0 @@
use crate::codec::encoder::Encoder;
use crate::codec::framed_impl::{FramedImpl, WriteFrame};
use tokio::{io::AsyncWrite, stream::Stream};
use futures_sink::Sink;
use pin_project_lite::pin_project;
use std::fmt;
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
pin_project! {
/// A [`Sink`] of frames encoded to an `AsyncWrite`.
///
/// [`Sink`]: futures_sink::Sink
pub struct FramedWrite<T, E> {
#[pin]
inner: FramedImpl<T, E, WriteFrame>,
}
}
impl<T, E> FramedWrite<T, E>
where
T: AsyncWrite,
{
/// Creates a new `FramedWrite` with the given `encoder`.
pub fn new(inner: T, encoder: E) -> FramedWrite<T, E> {
FramedWrite {
inner: FramedImpl {
inner,
codec: encoder,
state: WriteFrame::default(),
},
}
}
}
impl<T, E> FramedWrite<T, E> {
/// Returns a reference to the underlying I/O stream wrapped by
/// `FramedWrite`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_ref(&self) -> &T {
&self.inner.inner
}
/// Returns a mutable reference to the underlying I/O stream wrapped by
/// `FramedWrite`.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner.inner
}
/// Consumes the `FramedWrite`, returning its underlying I/O stream.
///
/// Note that care should be taken to not tamper with the underlying stream
/// of data coming in as it may corrupt the stream of frames otherwise
/// being worked with.
pub fn into_inner(self) -> T {
self.inner.inner
}
/// Returns a reference to the underlying encoder.
pub fn encoder(&self) -> &E {
&self.inner.codec
}
/// Returns a mutable reference to the underlying encoder.
pub fn encoder_mut(&mut self) -> &mut E {
&mut self.inner.codec
}
}
// This impl just defers to the underlying FramedImpl
impl<T, I, E> Sink<I> for FramedWrite<T, E>
where
T: AsyncWrite,
E: Encoder<I>,
E::Error: From<io::Error>,
{
type Error = E::Error;
fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.poll_ready(cx)
}
fn start_send(self: Pin<&mut Self>, item: I) -> Result<(), Self::Error> {
self.project().inner.start_send(item)
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.poll_flush(cx)
}
fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.project().inner.poll_close(cx)
}
}
// This impl just defers to the underlying T: Stream
impl<T, D> Stream for FramedWrite<T, D>
where
T: Stream,
{
type Item = T::Item;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.project().inner.project().inner.poll_next(cx)
}
}
impl<T, U> fmt::Debug for FramedWrite<T, U>
where
T: fmt::Debug,
U: fmt::Debug,
{
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("FramedWrite")
.field("inner", &self.get_ref())
.field("encoder", &self.encoder())
.field("buffer", &self.inner.state.buffer)
.finish()
}
}
-993
View File
@@ -1,993 +0,0 @@
//! Frame a stream of bytes based on a length prefix
//!
//! Many protocols delimit their frames by prefacing frame data with a
//! frame head that specifies the length of the frame. The
//! `length_delimited` module provides utilities for handling the length
//! based framing. This allows the consumer to work with entire frames
//! without having to worry about buffering or other framing logic.
//!
//! # Getting started
//!
//! If implementing a protocol from scratch, using length delimited framing
//! is an easy way to get started. [`LengthDelimitedCodec::new()`] will
//! return a length delimited codec using default configuration values.
//! This can then be used to construct a framer to adapt a full-duplex
//! byte stream into a stream of frames.
//!
//! ```
//! use tokio::io::{AsyncRead, AsyncWrite};
//! use tokio_util::codec::{Framed, LengthDelimitedCodec};
//!
//! fn bind_transport<T: AsyncRead + AsyncWrite>(io: T)
//! -> Framed<T, LengthDelimitedCodec>
//! {
//! Framed::new(io, LengthDelimitedCodec::new())
//! }
//! # pub fn main() {}
//! ```
//!
//! The returned transport implements `Sink + Stream` for `BytesMut`. It
//! encodes the frame with a big-endian `u32` header denoting the frame
//! payload length:
//!
//! ```text
//! +----------+--------------------------------+
//! | len: u32 | frame payload |
//! +----------+--------------------------------+
//! ```
//!
//! Specifically, given the following:
//!
//! ```
//! use tokio::prelude::*;
//! use tokio_util::codec::{Framed, LengthDelimitedCodec};
//!
//! use futures::SinkExt;
//! use bytes::Bytes;
//!
//! async fn write_frame<T>(io: T) -> Result<(), Box<dyn std::error::Error>>
//! where
//! T: AsyncRead + AsyncWrite + Unpin,
//! {
//! let mut transport = Framed::new(io, LengthDelimitedCodec::new());
//! let frame = Bytes::from("hello world");
//!
//! transport.send(frame).await?;
//! Ok(())
//! }
//! ```
//!
//! The encoded frame will look like this:
//!
//! ```text
//! +---- len: u32 ----+---- data ----+
//! | \x00\x00\x00\x0b | hello world |
//! +------------------+--------------+
//! ```
//!
//! # Decoding
//!
//! [`FramedRead`] adapts an [`AsyncRead`] into a `Stream` of [`BytesMut`],
//! such that each yielded [`BytesMut`] value contains the contents of an
//! entire frame. There are many configuration parameters enabling
//! [`FramedRead`] to handle a wide range of protocols. Here are some
//! examples that will cover the various options at a high level.
//!
//! ## Example 1
//!
//! The following will parse a `u16` length field at offset 0, including the
//! frame head in the yielded `BytesMut`.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(0) // default value
//! .length_field_length(2)
//! .length_adjustment(0) // default value
//! .num_skip(0) // Do not strip frame header
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +-- len ---+--- Payload ---+ +-- len ---+--- Payload ---+
//! | \x00\x0B | Hello world | --> | \x00\x0B | Hello world |
//! +----------+---------------+ +----------+---------------+
//! ```
//!
//! The value of the length field is 11 (`\x0B`) which represents the length
//! of the payload, `hello world`. By default, [`FramedRead`] assumes that
//! the length field represents the number of bytes that **follows** the
//! length field. Thus, the entire frame has a length of 13: 2 bytes for the
//! frame head + 11 bytes for the payload.
//!
//! ## Example 2
//!
//! The following will parse a `u16` length field at offset 0, omitting the
//! frame head in the yielded `BytesMut`.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(0) // default value
//! .length_field_length(2)
//! .length_adjustment(0) // default value
//! // `num_skip` is not needed, the default is to skip
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +-- len ---+--- Payload ---+ +--- Payload ---+
//! | \x00\x0B | Hello world | --> | Hello world |
//! +----------+---------------+ +---------------+
//! ```
//!
//! This is similar to the first example, the only difference is that the
//! frame head is **not** included in the yielded `BytesMut` value.
//!
//! ## Example 3
//!
//! The following will parse a `u16` length field at offset 0, including the
//! frame head in the yielded `BytesMut`. In this case, the length field
//! **includes** the frame head length.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(0) // default value
//! .length_field_length(2)
//! .length_adjustment(-2) // size of head
//! .num_skip(0)
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +-- len ---+--- Payload ---+ +-- len ---+--- Payload ---+
//! | \x00\x0D | Hello world | --> | \x00\x0D | Hello world |
//! +----------+---------------+ +----------+---------------+
//! ```
//!
//! In most cases, the length field represents the length of the payload
//! only, as shown in the previous examples. However, in some protocols the
//! length field represents the length of the whole frame, including the
//! head. In such cases, we specify a negative `length_adjustment` to adjust
//! the value provided in the frame head to represent the payload length.
//!
//! ## Example 4
//!
//! The following will parse a 3 byte length field at offset 0 in a 5 byte
//! frame head, including the frame head in the yielded `BytesMut`.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(0) // default value
//! .length_field_length(3)
//! .length_adjustment(2) // remaining head
//! .num_skip(0)
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT
//! +---- len -----+- head -+--- Payload ---+
//! | \x00\x00\x0B | \xCAFE | Hello world |
//! +--------------+--------+---------------+
//!
//! DECODED
//! +---- len -----+- head -+--- Payload ---+
//! | \x00\x00\x0B | \xCAFE | Hello world |
//! +--------------+--------+---------------+
//! ```
//!
//! A more advanced example that shows a case where there is extra frame
//! head data between the length field and the payload. In such cases, it is
//! usually desirable to include the frame head as part of the yielded
//! `BytesMut`. This lets consumers of the length delimited framer to
//! process the frame head as needed.
//!
//! The positive `length_adjustment` value lets `FramedRead` factor in the
//! additional head into the frame length calculation.
//!
//! ## Example 5
//!
//! The following will parse a `u16` length field at offset 1 of a 4 byte
//! frame head. The first byte and the length field will be omitted from the
//! yielded `BytesMut`, but the trailing 2 bytes of the frame head will be
//! included.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(1) // length of hdr1
//! .length_field_length(2)
//! .length_adjustment(1) // length of hdr2
//! .num_skip(3) // length of hdr1 + LEN
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT
//! +- hdr1 -+-- len ---+- hdr2 -+--- Payload ---+
//! | \xCA | \x00\x0B | \xFE | Hello world |
//! +--------+----------+--------+---------------+
//!
//! DECODED
//! +- hdr2 -+--- Payload ---+
//! | \xFE | Hello world |
//! +--------+---------------+
//! ```
//!
//! The length field is situated in the middle of the frame head. In this
//! case, the first byte in the frame head could be a version or some other
//! identifier that is not needed for processing. On the other hand, the
//! second half of the head is needed.
//!
//! `length_field_offset` indicates how many bytes to skip before starting
//! to read the length field. `length_adjustment` is the number of bytes to
//! skip starting at the end of the length field. In this case, it is the
//! second half of the head.
//!
//! ## Example 6
//!
//! The following will parse a `u16` length field at offset 1 of a 4 byte
//! frame head. The first byte and the length field will be omitted from the
//! yielded `BytesMut`, but the trailing 2 bytes of the frame head will be
//! included. In this case, the length field **includes** the frame head
//! length.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(1) // length of hdr1
//! .length_field_length(2)
//! .length_adjustment(-3) // length of hdr1 + LEN, negative
//! .num_skip(3)
//! .new_read(io);
//! # }
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT
//! +- hdr1 -+-- len ---+- hdr2 -+--- Payload ---+
//! | \xCA | \x00\x0F | \xFE | Hello world |
//! +--------+----------+--------+---------------+
//!
//! DECODED
//! +- hdr2 -+--- Payload ---+
//! | \xFE | Hello world |
//! +--------+---------------+
//! ```
//!
//! Similar to the example above, the difference is that the length field
//! represents the length of the entire frame instead of just the payload.
//! The length of `hdr1` and `len` must be counted in `length_adjustment`.
//! Note that the length of `hdr2` does **not** need to be explicitly set
//! anywhere because it already is factored into the total frame length that
//! is read from the byte stream.
//!
//! ## Example 7
//!
//! The following will parse a 3 byte length field at offset 0 in a 4 byte
//! frame head, excluding the 4th byte from the yielded `BytesMut`.
//!
//! ```
//! # use tokio::io::AsyncRead;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! LengthDelimitedCodec::builder()
//! .length_field_offset(0) // default value
//! .length_field_length(3)
//! .length_adjustment(0) // default value
//! .num_skip(4) // skip the first 4 bytes
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +------- len ------+--- Payload ---+ +--- Payload ---+
//! | \x00\x00\x0B\xFF | Hello world | => | Hello world |
//! +------------------+---------------+ +---------------+
//! ```
//!
//! A simple example where there are unused bytes between the length field
//! and the payload.
//!
//! # Encoding
//!
//! [`FramedWrite`] adapts an [`AsyncWrite`] into a `Sink` of [`BytesMut`],
//! such that each submitted [`BytesMut`] is prefaced by a length field.
//! There are fewer configuration options than [`FramedRead`]. Given
//! protocols that have more complex frame heads, an encoder should probably
//! be written by hand using [`Encoder`].
//!
//! Here is a simple example, given a `FramedWrite` with the following
//! configuration:
//!
//! ```
//! # use tokio::io::AsyncWrite;
//! # use tokio_util::codec::LengthDelimitedCodec;
//! # fn write_frame<T: AsyncWrite>(io: T) {
//! # let _ =
//! LengthDelimitedCodec::builder()
//! .length_field_length(2)
//! .new_write(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! A payload of `hello world` will be encoded as:
//!
//! ```text
//! +- len: u16 -+---- data ----+
//! | \x00\x0b | hello world |
//! +------------+--------------+
//! ```
//!
//! [`LengthDelimitedCodec::new()`]: method@LengthDelimitedCodec::new
//! [`FramedRead`]: struct@FramedRead
//! [`FramedWrite`]: struct@FramedWrite
//! [`AsyncRead`]: trait@tokio::io::AsyncRead
//! [`AsyncWrite`]: trait@tokio::io::AsyncWrite
//! [`Encoder`]: trait@Encoder
//! [`BytesMut`]: bytes::BytesMut
use crate::codec::{Decoder, Encoder, Framed, FramedRead, FramedWrite};
use tokio::io::{AsyncRead, AsyncWrite};
use bytes::{Buf, BufMut, Bytes, BytesMut};
use std::error::Error as StdError;
use std::io::{self, Cursor};
use std::{cmp, fmt};
/// Configure length delimited `LengthDelimitedCodec`s.
///
/// `Builder` enables constructing configured length delimited codecs. Note
/// that not all configuration settings apply to both encoding and decoding. See
/// the documentation for specific methods for more detail.
#[derive(Debug, Clone, Copy)]
pub struct Builder {
// Maximum frame length
max_frame_len: usize,
// Number of bytes representing the field length
length_field_len: usize,
// Number of bytes in the header before the length field
length_field_offset: usize,
// Adjust the length specified in the header field by this amount
length_adjustment: isize,
// Total number of bytes to skip before reading the payload, if not set,
// `length_field_len + length_field_offset`
num_skip: Option<usize>,
// Length field byte order (little or big endian)
length_field_is_big_endian: bool,
}
/// An error when the number of bytes read is more than max frame length.
pub struct LengthDelimitedCodecError {
_priv: (),
}
/// A codec for frames delimited by a frame head specifying their lengths.
///
/// This allows the consumer to work with entire frames without having to worry
/// about buffering or other framing logic.
///
/// See [module level] documentation for more detail.
///
/// [module level]: index.html
#[derive(Debug)]
pub struct LengthDelimitedCodec {
// Configuration values
builder: Builder,
// Read state
state: DecodeState,
}
#[derive(Debug, Clone, Copy)]
enum DecodeState {
Head,
Data(usize),
}
// ===== impl LengthDelimitedCodec ======
impl LengthDelimitedCodec {
/// Creates a new `LengthDelimitedCodec` with the default configuration values.
pub fn new() -> Self {
Self {
builder: Builder::new(),
state: DecodeState::Head,
}
}
/// Creates a new length delimited codec builder with default configuration
/// values.
pub fn builder() -> Builder {
Builder::new()
}
/// Returns the current max frame setting
///
/// This is the largest size this codec will accept from the wire. Larger
/// frames will be rejected.
pub fn max_frame_length(&self) -> usize {
self.builder.max_frame_len
}
/// Updates the max frame setting.
///
/// The change takes effect the next time a frame is decoded. In other
/// words, if a frame is currently in process of being decoded with a frame
/// size greater than `val` but less than the max frame length in effect
/// before calling this function, then the frame will be allowed.
pub fn set_max_frame_length(&mut self, val: usize) {
self.builder.max_frame_length(val);
}
fn decode_head(&mut self, src: &mut BytesMut) -> io::Result<Option<usize>> {
let head_len = self.builder.num_head_bytes();
let field_len = self.builder.length_field_len;
if src.len() < head_len {
// Not enough data
return Ok(None);
}
let n = {
let mut src = Cursor::new(&mut *src);
// Skip the required bytes
src.advance(self.builder.length_field_offset);
// match endianess
let n = if self.builder.length_field_is_big_endian {
src.get_uint(field_len)
} else {
src.get_uint_le(field_len)
};
if n > self.builder.max_frame_len as u64 {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
LengthDelimitedCodecError { _priv: () },
));
}
// The check above ensures there is no overflow
let n = n as usize;
// Adjust `n` with bounds checking
let n = if self.builder.length_adjustment < 0 {
n.checked_sub(-self.builder.length_adjustment as usize)
} else {
n.checked_add(self.builder.length_adjustment as usize)
};
// Error handling
match n {
Some(n) => n,
None => {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"provided length would overflow after adjustment",
));
}
}
};
let num_skip = self.builder.get_num_skip();
if num_skip > 0 {
src.advance(num_skip);
}
// Ensure that the buffer has enough space to read the incoming
// payload
src.reserve(n);
Ok(Some(n))
}
fn decode_data(&self, n: usize, src: &mut BytesMut) -> io::Result<Option<BytesMut>> {
// At this point, the buffer has already had the required capacity
// reserved. All there is to do is read.
if src.len() < n {
return Ok(None);
}
Ok(Some(src.split_to(n)))
}
}
impl Decoder for LengthDelimitedCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, src: &mut BytesMut) -> io::Result<Option<BytesMut>> {
let n = match self.state {
DecodeState::Head => match self.decode_head(src)? {
Some(n) => {
self.state = DecodeState::Data(n);
n
}
None => return Ok(None),
},
DecodeState::Data(n) => n,
};
match self.decode_data(n, src)? {
Some(data) => {
// Update the decode state
self.state = DecodeState::Head;
// Make sure the buffer has enough space to read the next head
src.reserve(self.builder.num_head_bytes());
Ok(Some(data))
}
None => Ok(None),
}
}
}
impl Encoder<Bytes> for LengthDelimitedCodec {
type Error = io::Error;
fn encode(&mut self, data: Bytes, dst: &mut BytesMut) -> Result<(), io::Error> {
let n = data.len();
if n > self.builder.max_frame_len {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
LengthDelimitedCodecError { _priv: () },
));
}
// Adjust `n` with bounds checking
let n = if self.builder.length_adjustment < 0 {
n.checked_add(-self.builder.length_adjustment as usize)
} else {
n.checked_sub(self.builder.length_adjustment as usize)
};
let n = n.ok_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"provided length would overflow after adjustment",
)
})?;
// Reserve capacity in the destination buffer to fit the frame and
// length field (plus adjustment).
dst.reserve(self.builder.length_field_len + n);
if self.builder.length_field_is_big_endian {
dst.put_uint(n as u64, self.builder.length_field_len);
} else {
dst.put_uint_le(n as u64, self.builder.length_field_len);
}
// Write the frame to the buffer
dst.extend_from_slice(&data[..]);
Ok(())
}
}
impl Default for LengthDelimitedCodec {
fn default() -> Self {
Self::new()
}
}
// ===== impl Builder =====
impl Builder {
/// Creates a new length delimited codec builder with default configuration
/// values.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .length_field_offset(0)
/// .length_field_length(2)
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new() -> Builder {
Builder {
// Default max frame length of 8MB
max_frame_len: 8 * 1_024 * 1_024,
// Default byte length of 4
length_field_len: 4,
// Default to the header field being at the start of the header.
length_field_offset: 0,
length_adjustment: 0,
// Total number of bytes to skip before reading the payload, if not set,
// `length_field_len + length_field_offset`
num_skip: None,
// Default to reading the length field in network (big) endian.
length_field_is_big_endian: true,
}
}
/// Read the length field as a big endian integer
///
/// This is the default setting.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .big_endian()
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn big_endian(&mut self) -> &mut Self {
self.length_field_is_big_endian = true;
self
}
/// Read the length field as a little endian integer
///
/// The default setting is big endian.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .little_endian()
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn little_endian(&mut self) -> &mut Self {
self.length_field_is_big_endian = false;
self
}
/// Read the length field as a native endian integer
///
/// The default setting is big endian.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .native_endian()
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn native_endian(&mut self) -> &mut Self {
if cfg!(target_endian = "big") {
self.big_endian()
} else {
self.little_endian()
}
}
/// Sets the max frame length
///
/// This configuration option applies to both encoding and decoding. The
/// default value is 8MB.
///
/// When decoding, the length field read from the byte stream is checked
/// against this setting **before** any adjustments are applied. When
/// encoding, the length of the submitted payload is checked against this
/// setting.
///
/// When frames exceed the max length, an `io::Error` with the custom value
/// of the `LengthDelimitedCodecError` type will be returned.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .max_frame_length(8 * 1024)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn max_frame_length(&mut self, val: usize) -> &mut Self {
self.max_frame_len = val;
self
}
/// Sets the number of bytes used to represent the length field
///
/// The default value is `4`. The max value is `8`.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .length_field_length(4)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn length_field_length(&mut self, val: usize) -> &mut Self {
assert!(val > 0 && val <= 8, "invalid length field length");
self.length_field_len = val;
self
}
/// Sets the number of bytes in the header before the length field
///
/// This configuration option only applies to decoding.
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .length_field_offset(1)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn length_field_offset(&mut self, val: usize) -> &mut Self {
self.length_field_offset = val;
self
}
/// Delta between the payload length specified in the header and the real
/// payload length
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .length_adjustment(-2)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn length_adjustment(&mut self, val: isize) -> &mut Self {
self.length_adjustment = val;
self
}
/// Sets the number of bytes to skip before reading the payload
///
/// Default value is `length_field_len + length_field_offset`
///
/// This configuration option only applies to decoding
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .num_skip(4)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn num_skip(&mut self, val: usize) -> &mut Self {
self.num_skip = Some(val);
self
}
/// Create a configured length delimited `LengthDelimitedCodec`
///
/// # Examples
///
/// ```
/// use tokio_util::codec::LengthDelimitedCodec;
/// # pub fn main() {
/// LengthDelimitedCodec::builder()
/// .length_field_offset(0)
/// .length_field_length(2)
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_codec();
/// # }
/// ```
pub fn new_codec(&self) -> LengthDelimitedCodec {
LengthDelimitedCodec {
builder: *self,
state: DecodeState::Head,
}
}
/// Create a configured length delimited `FramedRead`
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// .length_field_offset(0)
/// .length_field_length(2)
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
where
T: AsyncRead,
{
FramedRead::new(upstream, self.new_codec())
}
/// Create a configured length delimited `FramedWrite`
///
/// # Examples
///
/// ```
/// # use tokio::io::AsyncWrite;
/// # use tokio_util::codec::LengthDelimitedCodec;
/// # fn write_frame<T: AsyncWrite>(io: T) {
/// LengthDelimitedCodec::builder()
/// .length_field_length(2)
/// .new_write(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new_write<T>(&self, inner: T) -> FramedWrite<T, LengthDelimitedCodec>
where
T: AsyncWrite,
{
FramedWrite::new(inner, self.new_codec())
}
/// Create a configured length delimited `Framed`
///
/// # Examples
///
/// ```
/// # use tokio::io::{AsyncRead, AsyncWrite};
/// # use tokio_util::codec::LengthDelimitedCodec;
/// # fn write_frame<T: AsyncRead + AsyncWrite>(io: T) {
/// # let _ =
/// LengthDelimitedCodec::builder()
/// .length_field_length(2)
/// .new_framed(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new_framed<T>(&self, inner: T) -> Framed<T, LengthDelimitedCodec>
where
T: AsyncRead + AsyncWrite,
{
Framed::new(inner, self.new_codec())
}
fn num_head_bytes(&self) -> usize {
let num = self.length_field_offset + self.length_field_len;
cmp::max(num, self.num_skip.unwrap_or(0))
}
fn get_num_skip(&self) -> usize {
self.num_skip
.unwrap_or(self.length_field_offset + self.length_field_len)
}
}
impl Default for Builder {
fn default() -> Self {
Self::new()
}
}
// ===== impl LengthDelimitedCodecError =====
impl fmt::Debug for LengthDelimitedCodecError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("LengthDelimitedCodecError").finish()
}
}
impl fmt::Display for LengthDelimitedCodecError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str("frame size too big")
}
}
impl StdError for LengthDelimitedCodecError {}
-230
View File
@@ -1,230 +0,0 @@
use crate::codec::decoder::Decoder;
use crate::codec::encoder::Encoder;
use bytes::{Buf, BufMut, BytesMut};
use std::{cmp, fmt, io, str, usize};
/// A simple [`Decoder`] and [`Encoder`] implementation that splits up data into lines.
///
/// [`Decoder`]: crate::codec::Decoder
/// [`Encoder`]: crate::codec::Encoder
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
pub struct LinesCodec {
// Stored index of the next index to examine for a `\n` character.
// This is used to optimize searching.
// For example, if `decode` was called with `abc`, it would hold `3`,
// because that is the next index to examine.
// The next time `decode` is called with `abcde\n`, the method will
// only look at `de\n` before returning.
next_index: usize,
/// The maximum length for a given line. If `usize::MAX`, lines will be
/// read until a `\n` character is reached.
max_length: usize,
/// Are we currently discarding the remainder of a line which was over
/// the length limit?
is_discarding: bool,
}
impl LinesCodec {
/// Returns a `LinesCodec` for splitting up data into lines.
///
/// # Note
///
/// The returned `LinesCodec` will not have an upper bound on the length
/// of a buffered line. See the documentation for [`new_with_max_length`]
/// for information on why this could be a potential security risk.
///
/// [`new_with_max_length`]: crate::codec::LinesCodec::new_with_max_length()
pub fn new() -> LinesCodec {
LinesCodec {
next_index: 0,
max_length: usize::MAX,
is_discarding: false,
}
}
/// Returns a `LinesCodec` with a maximum line length limit.
///
/// If this is set, calls to `LinesCodec::decode` will return a
/// [`LinesCodecError`] when a line exceeds the length limit. Subsequent calls
/// will discard up to `limit` bytes from that line until a newline
/// character is reached, returning `None` until the line over the limit
/// has been fully discarded. After that point, calls to `decode` will
/// function as normal.
///
/// # Note
///
/// Setting a length limit is highly recommended for any `LinesCodec` which
/// will be exposed to untrusted input. Otherwise, the size of the buffer
/// that holds the line currently being read is unbounded. An attacker could
/// exploit this unbounded buffer by sending an unbounded amount of input
/// without any `\n` characters, causing unbounded memory consumption.
///
/// [`LinesCodecError`]: crate::codec::LinesCodecError
pub fn new_with_max_length(max_length: usize) -> Self {
LinesCodec {
max_length,
..LinesCodec::new()
}
}
/// Returns the maximum line length when decoding.
///
/// ```
/// use std::usize;
/// use tokio_util::codec::LinesCodec;
///
/// let codec = LinesCodec::new();
/// assert_eq!(codec.max_length(), usize::MAX);
/// ```
/// ```
/// use tokio_util::codec::LinesCodec;
///
/// let codec = LinesCodec::new_with_max_length(256);
/// assert_eq!(codec.max_length(), 256);
/// ```
pub fn max_length(&self) -> usize {
self.max_length
}
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf)
.map_err(|_| io::Error::new(io::ErrorKind::InvalidData, "Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
if let Some(&b'\r') = s.last() {
&s[..s.len() - 1]
} else {
s
}
}
impl Decoder for LinesCodec {
type Item = String;
type Error = LinesCodecError;
fn decode(&mut self, buf: &mut BytesMut) -> Result<Option<String>, LinesCodecError> {
loop {
// Determine how far into the buffer we'll search for a newline. If
// there's no max_length set, we'll read to the end of the buffer.
let read_to = cmp::min(self.max_length.saturating_add(1), buf.len());
let newline_offset = buf[self.next_index..read_to]
.iter()
.position(|b| *b == b'\n');
match (self.is_discarding, newline_offset) {
(true, Some(offset)) => {
// If we found a newline, discard up to that offset and
// then stop discarding. On the next iteration, we'll try
// to read a line normally.
buf.advance(offset + self.next_index + 1);
self.is_discarding = false;
self.next_index = 0;
}
(true, None) => {
// Otherwise, we didn't find a newline, so we'll discard
// everything we read. On the next iteration, we'll continue
// discarding up to max_len bytes unless we find a newline.
buf.advance(read_to);
self.next_index = 0;
if buf.is_empty() {
return Err(LinesCodecError::MaxLineLengthExceeded);
}
}
(false, Some(offset)) => {
// Found a line!
let newline_index = offset + self.next_index;
self.next_index = 0;
let line = buf.split_to(newline_index + 1);
let line = &line[..line.len() - 1];
let line = without_carriage_return(line);
let line = utf8(line)?;
return Ok(Some(line.to_string()));
}
(false, None) if buf.len() > self.max_length => {
// Reached the maximum length without finding a
// newline, return an error and start discarding on the
// next call.
self.is_discarding = true;
return Err(LinesCodecError::MaxLineLengthExceeded);
}
(false, None) => {
// We didn't find a line or reach the length limit, so the next
// call will resume searching at the current offset.
self.next_index = read_to;
return Ok(None);
}
}
}
}
fn decode_eof(&mut self, buf: &mut BytesMut) -> Result<Option<String>, LinesCodecError> {
Ok(match self.decode(buf)? {
Some(frame) => Some(frame),
None => {
// No terminating newline - return remaining data, if any
if buf.is_empty() || buf == &b"\r"[..] {
None
} else {
let line = buf.split_to(buf.len());
let line = without_carriage_return(&line);
let line = utf8(line)?;
self.next_index = 0;
Some(line.to_string())
}
}
})
}
}
impl<T> Encoder<T> for LinesCodec
where
T: AsRef<str>,
{
type Error = LinesCodecError;
fn encode(&mut self, line: T, buf: &mut BytesMut) -> Result<(), LinesCodecError> {
let line = line.as_ref();
buf.reserve(line.len() + 1);
buf.put(line.as_bytes());
buf.put_u8(b'\n');
Ok(())
}
}
impl Default for LinesCodec {
fn default() -> Self {
Self::new()
}
}
/// An error occured while encoding or decoding a line.
#[derive(Debug)]
pub enum LinesCodecError {
/// The maximum line length was exceeded.
MaxLineLengthExceeded,
/// An IO error occured.
Io(io::Error),
}
impl fmt::Display for LinesCodecError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
LinesCodecError::MaxLineLengthExceeded => write!(f, "max line length exceeded"),
LinesCodecError::Io(e) => write!(f, "{}", e),
}
}
}
impl From<io::Error> for LinesCodecError {
fn from(e: io::Error) -> LinesCodecError {
LinesCodecError::Io(e)
}
}
impl std::error::Error for LinesCodecError {}
-43
View File
@@ -1,43 +0,0 @@
//! Adaptors from AsyncRead/AsyncWrite to Stream/Sink
//!
//! Raw I/O objects work with byte sequences, but higher-level code
//! usually wants to batch these into meaningful chunks, called
//! "frames".
//!
//! This module contains adapters to go from streams of bytes,
//! [`AsyncRead`] and [`AsyncWrite`], to framed streams implementing
//! [`Sink`] and [`Stream`]. Framed streams are also known as
//! transports.
//!
//! [`AsyncRead`]: tokio::io::AsyncRead
//! [`AsyncWrite`]: tokio::io::AsyncWrite
//! [`Stream`]: tokio::stream::Stream
//! [`Sink`]: futures_sink::Sink
mod bytes_codec;
pub use self::bytes_codec::BytesCodec;
mod decoder;
pub use self::decoder::Decoder;
mod encoder;
pub use self::encoder::Encoder;
mod framed_impl;
#[allow(unused_imports)]
pub(crate) use self::framed_impl::{FramedImpl, RWFrames, ReadFrame, WriteFrame};
mod framed;
pub use self::framed::{Framed, FramedParts};
mod framed_read;
pub use self::framed_read::FramedRead;
mod framed_write;
pub use self::framed_write::FramedWrite;
pub mod length_delimited;
pub use self::length_delimited::{LengthDelimitedCodec, LengthDelimitedCodecError};
mod lines_codec;
pub use self::lines_codec::{LinesCodec, LinesCodecError};
-217
View File
@@ -1,217 +0,0 @@
//! Compatibility between the `tokio::io` and `futures-io` versions of the
//! `AsyncRead` and `AsyncWrite` traits.
use futures_core::ready;
use pin_project_lite::pin_project;
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
pin_project! {
/// A compatibility layer that allows conversion between the
/// `tokio::io` and `futures-io` `AsyncRead` and `AsyncWrite` traits.
#[derive(Copy, Clone, Debug)]
pub struct Compat<T> {
#[pin]
inner: T,
}
}
/// Extension trait that allows converting a type implementing
/// `futures_io::AsyncRead` to implement `tokio::io::AsyncRead`.
pub trait FuturesAsyncReadCompatExt: futures_io::AsyncRead {
/// Wraps `self` with a compatibility layer that implements
/// `tokio_io::AsyncRead`.
fn compat(self) -> Compat<Self>
where
Self: Sized,
{
Compat::new(self)
}
}
impl<T: futures_io::AsyncRead> FuturesAsyncReadCompatExt for T {}
/// Extension trait that allows converting a type implementing
/// `futures_io::AsyncWrite` to implement `tokio::io::AsyncWrite`.
pub trait FuturesAsyncWriteCompatExt: futures_io::AsyncWrite {
/// Wraps `self` with a compatibility layer that implements
/// `tokio::io::AsyncWrite`.
fn compat_write(self) -> Compat<Self>
where
Self: Sized,
{
Compat::new(self)
}
}
impl<T: futures_io::AsyncWrite> FuturesAsyncWriteCompatExt for T {}
/// Extension trait that allows converting a type implementing
/// `tokio::io::AsyncRead` to implement `futures_io::AsyncRead`.
pub trait Tokio02AsyncReadCompatExt: tokio::io::AsyncRead {
/// Wraps `self` with a compatibility layer that implements
/// `futures_io::AsyncRead`.
fn compat(self) -> Compat<Self>
where
Self: Sized,
{
Compat::new(self)
}
}
impl<T: tokio::io::AsyncRead> Tokio02AsyncReadCompatExt for T {}
/// Extension trait that allows converting a type implementing
/// `tokio::io::AsyncWrite` to implement `futures_io::AsyncWrite`.
pub trait Tokio02AsyncWriteCompatExt: tokio::io::AsyncWrite {
/// Wraps `self` with a compatibility layer that implements
/// `futures_io::AsyncWrite`.
fn compat_write(self) -> Compat<Self>
where
Self: Sized,
{
Compat::new(self)
}
}
impl<T: tokio::io::AsyncWrite> Tokio02AsyncWriteCompatExt for T {}
// === impl Compat ===
impl<T> Compat<T> {
fn new(inner: T) -> Self {
Self { inner }
}
/// Get a reference to the `Future`, `Stream`, `AsyncRead`, or `AsyncWrite` object
/// contained within.
pub fn get_ref(&self) -> &T {
&self.inner
}
/// Get a mutable reference to the `Future`, `Stream`, `AsyncRead`, or `AsyncWrite` object
/// contained within.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
/// Returns the wrapped item.
pub fn into_inner(self) -> T {
self.inner
}
}
impl<T> tokio::io::AsyncRead for Compat<T>
where
T: futures_io::AsyncRead,
{
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>,
) -> Poll<io::Result<()>> {
// We can't trust the inner type to not peak at the bytes,
// so we must defensively initialize the buffer.
let slice = buf.initialize_unfilled();
let n = ready!(futures_io::AsyncRead::poll_read(
self.project().inner,
cx,
slice
))?;
buf.advance(n);
Poll::Ready(Ok(()))
}
}
impl<T> futures_io::AsyncRead for Compat<T>
where
T: tokio::io::AsyncRead,
{
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
slice: &mut [u8],
) -> Poll<io::Result<usize>> {
let mut buf = tokio::io::ReadBuf::new(slice);
ready!(tokio::io::AsyncRead::poll_read(
self.project().inner,
cx,
&mut buf
))?;
Poll::Ready(Ok(buf.filled().len()))
}
}
impl<T> tokio::io::AsyncBufRead for Compat<T>
where
T: futures_io::AsyncBufRead,
{
fn poll_fill_buf<'a>(
self: Pin<&'a mut Self>,
cx: &mut Context<'_>,
) -> Poll<io::Result<&'a [u8]>> {
futures_io::AsyncBufRead::poll_fill_buf(self.project().inner, cx)
}
fn consume(self: Pin<&mut Self>, amt: usize) {
futures_io::AsyncBufRead::consume(self.project().inner, amt)
}
}
impl<T> futures_io::AsyncBufRead for Compat<T>
where
T: tokio::io::AsyncBufRead,
{
fn poll_fill_buf<'a>(
self: Pin<&'a mut Self>,
cx: &mut Context<'_>,
) -> Poll<io::Result<&'a [u8]>> {
tokio::io::AsyncBufRead::poll_fill_buf(self.project().inner, cx)
}
fn consume(self: Pin<&mut Self>, amt: usize) {
tokio::io::AsyncBufRead::consume(self.project().inner, amt)
}
}
impl<T> tokio::io::AsyncWrite for Compat<T>
where
T: futures_io::AsyncWrite,
{
fn poll_write(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
futures_io::AsyncWrite::poll_write(self.project().inner, cx, buf)
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
futures_io::AsyncWrite::poll_flush(self.project().inner, cx)
}
fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
futures_io::AsyncWrite::poll_close(self.project().inner, cx)
}
}
impl<T> futures_io::AsyncWrite for Compat<T>
where
T: tokio::io::AsyncWrite,
{
fn poll_write(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
tokio::io::AsyncWrite::poll_write(self.project().inner, cx, buf)
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
tokio::io::AsyncWrite::poll_flush(self.project().inner, cx)
}
fn poll_close(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
tokio::io::AsyncWrite::poll_shutdown(self.project().inner, cx)
}
}
-78
View File
@@ -1,78 +0,0 @@
//! Tokio context aware futures utilities.
//!
//! This module includes utilities around integrating tokio with other runtimes
//! by allowing the context to be attached to futures. This allows spawning
//! futures on other executors while still using tokio to drive them. This
//! can be useful if you need to use a tokio based library in an executor/runtime
//! that does not provide a tokio context.
use pin_project_lite::pin_project;
use std::{
future::Future,
pin::Pin,
task::{Context, Poll},
};
use tokio::runtime::Runtime;
pin_project! {
/// `TokioContext` allows connecting a custom executor with the tokio runtime.
///
/// It contains a `Handle` to the runtime. A handle to the runtime can be
/// obtain by calling the `Runtime::handle()` method.
pub struct TokioContext<'a, F> {
#[pin]
inner: F,
handle: &'a Runtime,
}
}
impl<F: Future> Future for TokioContext<'_, F> {
type Output = F::Output;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
let me = self.project();
let handle = me.handle;
let fut = me.inner;
let _enter = handle.enter();
fut.poll(cx)
}
}
/// Trait extension that simplifies bundling a `Handle` with a `Future`.
pub trait RuntimeExt {
/// Convenience method that takes a Future and returns a `TokioContext`.
///
/// # Example: calling Tokio Runtime from a custom ThreadPool
///
/// ```no_run
/// use tokio_util::context::RuntimeExt;
/// use tokio::time::{sleep, Duration};
///
/// let rt = tokio::runtime::Builder::new_multi_thread()
/// .enable_all()
/// .build()
/// .unwrap();
///
/// let rt2 = tokio::runtime::Builder::new_multi_thread()
/// .build()
/// .unwrap();
///
/// let fut = sleep(Duration::from_millis(2));
///
/// rt.block_on(
/// rt2
/// .wrap(async { sleep(Duration::from_millis(2)).await }),
/// );
///```
fn wrap<F: Future>(&self, fut: F) -> TokioContext<'_, F>;
}
impl RuntimeExt for Runtime {
fn wrap<F: Future>(&self, fut: F) -> TokioContext<'_, F> {
TokioContext {
inner: fut,
handle: self,
}
}
}
-190
View File
@@ -1,190 +0,0 @@
//! Module defining an Either type.
use std::{
future::Future,
io::SeekFrom,
pin::Pin,
task::{Context, Poll},
};
use tokio::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf, Result};
/// Combines two different futures, streams, or sinks having the same associated types into a single type.
///
/// This type implements common asynchronous traits such as [`Future`] and those in Tokio.
///
/// [`Future`]: std::future::Future
///
/// # Example
///
/// The following code will not work:
///
/// ```compile_fail
/// # fn some_condition() -> bool { true }
/// # async fn some_async_function() -> u32 { 10 }
/// # async fn other_async_function() -> u32 { 20 }
/// #[tokio::main]
/// async fn main() {
/// let result = if some_condition() {
/// some_async_function()
/// } else {
/// other_async_function() // <- Will print: "`if` and `else` have incompatible types"
/// };
///
/// println!("Result is {}", result.await);
/// }
/// ```
///
// This is because although the output types for both futures is the same, the exact future
// types are different, but the compiler must be able to choose a single type for the
// `result` variable.
///
/// When the output type is the same, we can wrap each future in `Either` to avoid the
/// issue:
///
/// ```
/// use tokio_util::either::Either;
/// # fn some_condition() -> bool { true }
/// # async fn some_async_function() -> u32 { 10 }
/// # async fn other_async_function() -> u32 { 20 }
///
/// #[tokio::main]
/// async fn main() {
/// let result = if some_condition() {
/// Either::Left(some_async_function())
/// } else {
/// Either::Right(other_async_function())
/// };
///
/// let value = result.await;
/// println!("Result is {}", value);
/// # assert_eq!(value, 10);
/// }
/// ```
#[allow(missing_docs)] // Doc-comments for variants in this particular case don't make much sense.
#[derive(Debug, Clone)]
pub enum Either<L, R> {
Left(L),
Right(R),
}
/// A small helper macro which reduces amount of boilerplate in the actual trait method implementation.
/// It takes an invokation of method as an argument (e.g. `self.poll(cx)`), and redirects it to either
/// enum variant held in `self`.
macro_rules! delegate_call {
($self:ident.$method:ident($($args:ident),+)) => {
unsafe {
match $self.get_unchecked_mut() {
Self::Left(l) => Pin::new_unchecked(l).$method($($args),+),
Self::Right(r) => Pin::new_unchecked(r).$method($($args),+),
}
}
}
}
impl<L, R, O> Future for Either<L, R>
where
L: Future<Output = O>,
R: Future<Output = O>,
{
type Output = O;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
delegate_call!(self.poll(cx))
}
}
impl<L, R> AsyncRead for Either<L, R>
where
L: AsyncRead,
R: AsyncRead,
{
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>> {
delegate_call!(self.poll_read(cx, buf))
}
}
impl<L, R> AsyncBufRead for Either<L, R>
where
L: AsyncBufRead,
R: AsyncBufRead,
{
fn poll_fill_buf(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<&[u8]>> {
delegate_call!(self.poll_fill_buf(cx))
}
fn consume(self: Pin<&mut Self>, amt: usize) {
delegate_call!(self.consume(amt))
}
}
impl<L, R> AsyncSeek for Either<L, R>
where
L: AsyncSeek,
R: AsyncSeek,
{
fn start_seek(self: Pin<&mut Self>, position: SeekFrom) -> Result<()> {
delegate_call!(self.start_seek(position))
}
fn poll_complete(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<u64>> {
delegate_call!(self.poll_complete(cx))
}
}
impl<L, R> AsyncWrite for Either<L, R>
where
L: AsyncWrite,
R: AsyncWrite,
{
fn poll_write(self: Pin<&mut Self>, cx: &mut Context<'_>, buf: &[u8]) -> Poll<Result<usize>> {
delegate_call!(self.poll_write(cx, buf))
}
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<tokio::io::Result<()>> {
delegate_call!(self.poll_flush(cx))
}
fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<tokio::io::Result<()>> {
delegate_call!(self.poll_shutdown(cx))
}
}
impl<L, R> futures_core::stream::Stream for Either<L, R>
where
L: futures_core::stream::Stream,
R: futures_core::stream::Stream<Item = L::Item>,
{
type Item = L::Item;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
delegate_call!(self.poll_next(cx))
}
}
#[cfg(test)]
mod tests {
use super::*;
use tokio::{
io::{repeat, AsyncReadExt, Repeat},
stream::{once, Once, StreamExt},
};
#[tokio::test]
async fn either_is_stream() {
let mut either: Either<Once<u32>, Once<u32>> = Either::Left(once(1));
assert_eq!(Some(1u32), either.next().await);
}
#[tokio::test]
async fn either_is_async_read() {
let mut buffer = [0; 3];
let mut either: Either<Repeat, Repeat> = Either::Right(repeat(0b101));
either.read_exact(&mut buffer).await.unwrap();
assert_eq!(buffer, [0b101, 0b101, 0b101]);
}
}

Some files were not shown because too many files have changed in this diff Show More