* io: ensure ReadHalf/WriteHalf do not return WouldBlock directly
These facades were passing back WouldBlock when the internal BiLock
couldn't be acquired, which does not fit the intended behavior.
Signed-off-by: Toby Lawrence <[email protected]>
* io: pull from the local crate, not crates.io
## Motivation
Currently, there is a potential denial of service vulnerability in the
`lines` codec. Since there is no bound on the buffer that holds data
before it is split into a new line, an attacker could send an unbounded
amount of data without sending a `\n` character.
## Solution
This branch adds a `new_with_max_length` constructor for `LinesCodec`
that configures a limit on the maximum number of bytes per line. When
the limit is reached, the the overly long line will be discarded (in
`max_length`-sized increments until a newline character or the end of the
buffer is reached. It was also necessary to add some special-case logic
to avoid creating an empty line when the length limit is reached at the
character immediately _before_ a `\n` character.
Additionally, this branch adds new tests for this function, including a
test for changing the line limit in-flight.
## Notes
This branch makes the following changes from my original PR with
this change (#590):
- The whole too-long line is discarded at once in the first call to `decode`
that encounters it.
- Only one error is emitted per too-long line.
- Made all the changes requested by @carllerche in
https://github.com/tokio-rs/tokio/pull/590#issuecomment-420735023Fixes: #186
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
Currently, the `RUST_BACKTRACE` environment variable is set to `1` on
Travis CI builds:
https://github.com/tokio-rs/tokio/blob/0ca973a7ebc5b8a29beac1ccb6c73ef26ddcbf22/.travis.yml#L49
However, it's not set on AppVeyor. This can make debugging
Windows-specific CI failures challenging for developers on other
operating systems.
## Solution
This branch sets `RUST_BACKTRACE=1` on AppVeyor.
Signed-off-by: Eliza Weisman <[email protected]>
Since the CI runs all tests for all tokio crates, it is possible that a
sporadic failure in one crate can mask failures/successes of other
crates' tests.
Using the `--no-fail-fast` flag instructs cargo to run *all* tests
before failing the build. This will allow checking to see if any
relevant test cases still pass even if an unrelated test has failed.
* Don't use tokio-core any more for tests. That one brings tokio from
crates.io instead of the current workspace and two versions of that
don't want to cooperate.
* Guard unix-specific examples on windows.
* Leave CI setup to top-level directory.
* Originally reported in alexcrichton/tokio-process#42
* The root cause appears to be due to two different PollEvented
instances trying to consume readiness events from the same file
descriptor.
* Previously we would simply swallow any `AlreadyExists` errors when
attempting to register the pipe receiver with the event loop. I'm not
sure if this means the PollEvented wrapper wasn't fully registered to
receive events, or maybe there is a potential race condition with how
PollEvented consumes mio readiness events. Using a fresh/duplicate file
descriptor appears to mitigate the issue, however.
* I was also not able to reproduce the issue as an isolated test case so
there is no regression test available within this crate (but we can add
one in tokio-process)
* codec: add new constructor `with_max_length ` to `LinesCodec`
* codec: add security note to docs
Signed-off-by: Eliza Weisman <[email protected]>
* Fix Rust 1.25 compatibility
* codec: Fix incorrect line lengths in tests (and add assertions)
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix off-by-one error in lines codec
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix call to decode rather than decode_eof in test
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix incorrect LinesCodec::decode_max_line_length
This bug was introduced after the fix for the off-by-one error.
Fortunately, the doctests caught it.
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Minor style improvements
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Don't allow LinesCodec length limit to be set after construction
Signed-off-by: Eliza Weisman <[email protected]>
* codec: change LinesCodec to error and discard line when at max length
* codec: Fix build on Rust 1.25
The slice patterns syntax wasn't supported yet in that release.
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Add test for out-of-bounds index when peeking
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix out of bounds index
* codec: Fix incomplete comment
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Add test for line decoder buffer underrun
@jonhoo reported a panic in the call to `LocalKey::with`, which occurs
when the reactor is dropped in the middle of TLS teardown. This PR
changes the call to `LocalKey::try_with` and handles the case when the
thread-local value has already been destroyed.
* Added timeouts to all tests that were missing them
- any issue we have will likely result in deadlocks/starvation so its
best if all tests quickly timeout rather than require getting killed
or have the CI timeout itself
* Added a `support` module and put a bunch of helpers there to DRY the
tests
* As observed in alexcrichton/tokio-signal#38, Signal instances can starve based on the order
they are created in, and this ordering appears to be platform/OS
specific
* The crux of the issue is that we woud only *attempt* to broadcast any
pending signals if we successfully read out at least one byte from the
global pipe.
* For reasons unclear to me, the affected Signal instance would get
woken up after the signal handler writes to the global pipe, but it
would immediately hit a WouldBlock error and give up, bypassing the
broadcast attempt (even though the pending flag was correctly set).
- Maybe this has to do with OS specifics with how the bytes are
delivered (or not), or with some complex interaction with tokio and
the pipe registration. It seems fishy since strace logs didn't show
the signal handler pipe write fail either, but I'm all out of ideas
* The fix appears simple: unconditionally attempt to broadcast any
pending signals *any* time a Driver instance is woken up.
* Since we perform an atomic check for each pending signal, we know that
each (coalesced) signal broadcast will happen at most once. If we were
supuriously woken up and no signals were pending, then nothing will be
yielded to any pollers of Signal
* The down side is that since each Signal instance polls a Driver
instance, each poll to Signal will essentially perform N atomic
operations (N = number of signals we support) in an attempt to broadcast
any pending signals.
- However, we can revisit optimizing this better in the future
Fixesalexcrichton/tokio-signal#38
* We introduce a new global structure which keeps track of how many
signal streams have been registered with a given event loop (the event
loop is identified by its OS file descriptor)
* We only attempt to deregister our global evented pipe from any event
loop if and only if we are the last signal that was registered with it
* Currently, whenever a new signal stream is created we attempt to
register a global pipe with the event loop to drive events.
* We also (correctly) swallow any descriptor-already-registered errors
since the same pipe is always used
* However, we currently *deregister* the same global pipe *any time* a
Signal stream is dropped.
* This means that if 2 or more of Signal instances exist simultaneously
(even if listening for different signals) and one of them is dropped,
the remainder will starve (until any new signal is created again).
* Cargo runs each integration-style-test in its own process. Since the
tests use global data structures specific to the process, we should run
them in an isolated manner to avoid having cross-test interactions
* Fixesalexcrichton/tokio-signal#39
This allows tokio-signal to build with `-Z minimal-versions` - see
https://github.com/rust-lang/cargo/issues/5657#issuecomment-401110172
for more details.
Earlier versions depend on log 0.3.1, which itself depends on libc
0.1, which doesn't build on any post-1.0 version of rust.
