How the buffer is managed is often critical for performance. Not
taking care of it will be catastrophic for performance beyond the
initial buffer size with the current implementation (a loop of
`reserve(1)`).
Strategy was to
- copy the old codec code that was temporarily being stashed in `tokio-io`
- modify all the type signatures to use Pin, as literal a translation as possible
- fix up the tests likewise
This is intended just to get things compiling and passing tests. Beyond that there is surely
lots of refactoring that can be done to make things more idiomatic. The docs are unchanged.
Closes#1189
## Motivation
Currently, there is a potential denial of service vulnerability in the
`lines` codec. Since there is no bound on the buffer that holds data
before it is split into a new line, an attacker could send an unbounded
amount of data without sending a `\n` character.
## Solution
This branch adds a `new_with_max_length` constructor for `LinesCodec`
that configures a limit on the maximum number of bytes per line. When
the limit is reached, the the overly long line will be discarded (in
`max_length`-sized increments until a newline character or the end of the
buffer is reached. It was also necessary to add some special-case logic
to avoid creating an empty line when the length limit is reached at the
character immediately _before_ a `\n` character.
Additionally, this branch adds new tests for this function, including a
test for changing the line limit in-flight.
## Notes
This branch makes the following changes from my original PR with
this change (#590):
- The whole too-long line is discarded at once in the first call to `decode`
that encounters it.
- Only one error is emitted per too-long line.
- Made all the changes requested by @carllerche in
https://github.com/tokio-rs/tokio/pull/590#issuecomment-420735023Fixes: #186
Signed-off-by: Eliza Weisman <[email protected]>
* codec: add new constructor `with_max_length ` to `LinesCodec`
* codec: add security note to docs
Signed-off-by: Eliza Weisman <[email protected]>
* Fix Rust 1.25 compatibility
* codec: Fix incorrect line lengths in tests (and add assertions)
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix off-by-one error in lines codec
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix call to decode rather than decode_eof in test
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix incorrect LinesCodec::decode_max_line_length
This bug was introduced after the fix for the off-by-one error.
Fortunately, the doctests caught it.
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Minor style improvements
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Don't allow LinesCodec length limit to be set after construction
Signed-off-by: Eliza Weisman <[email protected]>
* codec: change LinesCodec to error and discard line when at max length
* codec: Fix build on Rust 1.25
The slice patterns syntax wasn't supported yet in that release.
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Add test for out-of-bounds index when peeking
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Fix out of bounds index
* codec: Fix incomplete comment
Signed-off-by: Eliza Weisman <[email protected]>
* codec: Add test for line decoder buffer underrun
The `lines_encoder` test is a copy/paste of `bytes_encoder` and not
testing the LinesCodec encoding at all. This updates the test to do
simple validations of the LinesCodec encoding.
Sean McArthur