Compare commits

...
82 Commits
Author SHA1 Message Date
Alice RyhlandGitHub ebc61e5af1 chore: prepare bytes v1.1.0 (#509) 2021-08-25 17:48:41 +02:00
Christopher HotchkissandGitHub 55e296850d Clarifying actions of clear and truncate. (#508) 2021-08-24 16:12:20 +02:00
Ian JacksonandGitHub 0e9fa0b602 impl From<Box<[u8]>> for Bytes (#504) 2021-08-24 12:42:22 +02:00
ee24be7fa0 ci: fetch cargo hack from github release (#507)
Co-authored-by: Taiki Endo <[email protected]>
2021-08-24 12:33:16 +02:00
Stepan KoltsovandGitHub 2697fa7a9d BufMut::put_bytes(self, val, cnt) (#487)
Equivalent to

```
for _ in 0..cnt {
    self.put_u8(val);
}
```

but may work faster.

Name and signature is chosen to be consistent with `ptr::write_bytes`.

Include three specializations:
* `Vec<u8>`
* `&mut [u8]`
* `BytesMut`

`BytesMut` and `&mut [u8]` specializations use `ptr::write`, `Vec<u8>`
specialization uses `Vec::resize`.
2021-08-09 02:43:53 +09:00
Stepan KoltsovandGitHub fa9cbf1258 Clarify BufPut::put_int behavior (#486)
* writes low bytes, discards high bytes
* panics if `nbytes` is greater than 8
2021-08-09 02:43:09 +09:00
Alice RyhlandGitHub ab8e3c01a8 Clarify BufMut allocation guarantees (#501) 2021-08-07 08:22:18 +02:00
Taiki EndoandGitHub f34dc5c3f9 Remove doc URLs (#498) 2021-08-07 02:06:57 +09:00
GbillouandGitHub baaf12d22a Keep capacity when unsplit on empty other buf (#502) 2021-07-05 16:46:17 +02:00
Taiki EndoandGitHub ed1d24e570 Use ubuntu-latest instead of ubuntu-16.04 (#497) 2021-05-23 23:09:26 +09:00
Taiki EndoandGitHub b89247c713 Update loom to 0.5 (#494) 2021-04-13 23:58:18 +09:00
NoahandGitHub 9c770188fd Fully inline BytesMut::new (#493) 2021-04-11 05:49:33 +09:00
Stepan KoltsovandGitHub b9eade12a5 Specialize copy_to_bytes for Chain and Take (#481)
Avoid allocation when `Take` or `Chain` is composed of `Bytes`
objects.

This works now for `Take`.

`Chain` it works if the requested bytes does not cross boundary
between `Chain` members.
2021-04-11 03:56:35 +09:00
Dan BurkertandGitHub 3d5624a452 Add inline tags to UninitSlice methods (#443)
This appears to be the primary cause of significant performance
regressions in the `prost` test suite in the 0.5 to 0.6 transition.  See
danburkert/prost#381.
2021-04-11 03:19:30 +09:00
Stepan KoltsovandGitHub 2428c152a6 Panic on integer overflow in Chain::remaining (#482)
Make it safer.
2021-02-16 12:44:33 -08:00
ZettrokeandGitHub 268f6f80b4 override put_slice for &mut [u8] (#483) 2021-02-15 16:35:01 -08:00
Alice RyhlandGitHub e4182808df Make bytes_mut -> chunk_mut rename more easily discoverable (#471) 2021-01-23 15:22:43 +09:00
Christopher BunnandGitHub 8daf43e9bd docs: fix broken Take link (#466) 2021-01-20 14:39:23 +09:00
Alice RyhlandGitHub 7b18c1c076 prepare 1.0.1 release (#460) 2021-01-11 18:07:46 +01:00
Ralf JungandGitHub df20a68356 use Box::into_raw instead of mem-forget-in-disguise (#458) 2020-12-31 15:07:28 +01:00
laizyandGitHub 8758a1aba5 add inline for Vec::put_slice (#459) 2020-12-31 12:34:39 +01:00
Ralf JungandGitHub 27a0f9ca6e CI: run test suite in Miri (#456) 2020-12-29 22:54:48 +01:00
Alice RyhlandGitHub ed71a7beb3 Fix deprecation warning (#457) 2020-12-29 22:46:39 +01:00
Carl LercheandGitHub 064ad9a1a0 chore: prepare v1.0.0 release (#453) 2020-12-22 15:30:20 -08:00
Taiki EndoandGitHub ed1d194660 deps: update loom to 0.4 (#452) 2020-12-22 10:30:50 +01:00
Arve KnudsenandGitHub e398b0a209 Update readme / changelog (#451) 2020-12-20 07:31:22 -08:00
Carl LercheandGitHub 06907f3e7b Rename Buf/BufMut, methods to chunk/chunk_mut (#450)
The `bytes()` / `bytes_mut()` name implies the method returns the full
set of bytes represented by `Buf`/`BufMut`. To rectify this, the methods
are renamed to `chunk()` and `chunk_mut()` to reflect the partial nature
of the returned byte slice.

`bytes_vectored()` is renamed `chunks_vectored()`.

Closes #447
2020-12-18 11:04:31 -08:00
Carl LercheandGitHub 54f5ced6c5 remove unused Buf implementation. (#449)
The implementation of `Buf` for `Option<[u8; 1]>` was added to support
`IntoBuf`. The `IntoBuf` trait has since been removed.

Closes #444
2020-12-16 21:51:13 -08:00
Carl LercheandGitHub bd78f19393 chore: prepare for v1.0.0 work (#448) 2020-12-12 08:25:14 -08:00
Ibraheem AhmedandGitHub 39c6d7a512 Upgrade bytes version from 0.5 to 0.6.0 in README (#441) 2020-11-02 00:10:51 +09:00
Carl LercheandGitHub b7f75829ab prepare v0.6.0 release (#440) 2020-10-20 16:08:25 -07:00
Carl LercheandGitHub 39de065a1e Add Buf::copy_to_bytes(len) (#439)
This method replaces `Buf::to_bytes()`, providing a method that copies a
subset of the remaining buffer into a `Bytes` value. As this is strictly
more flexible, `to_bytes()` is removed.

Fixes: #129, #398
2020-10-20 11:00:35 -07:00
Carl LercheandGitHub 5866839e45 use checked addition with range (#438) 2020-10-20 10:26:38 -07:00
Carl LercheandGitHub e0d8413d53 Switch BufMut::bytes_mut to&mut UninitSlice (#433)
The way BufMut uses MaybeUninit can lead to unsoundness. This replaces
MaybeUnit with a type owned by bytes so we can ensure the usage patterns
are sound.

Refs: #328
2020-10-19 15:48:23 -07:00
Matthias EinwagandGitHub 5a11c783ec De-emphasize Arc implementation in Bytes description (#436)
The previous description focussed a lot on the `Arc` based implementation
of `Bytes`. Given the vtable based implemetation, this is however not the
only valid implementation. This changes the description a bit in order
to de-emaphasize the `Arc` part, and to describe that other implementations
are possible.

This should also be necessary if the vtable gets public.
2020-10-19 12:32:20 -07:00
Carl LercheandGitHub 4724c7e8a0 remove new fns from combinator structs (#434)
This is not idiomatic.
2020-10-18 09:55:19 -07:00
Carl LercheandGitHub ced050730c Make BufMut an unsafe trait (#432)
Users of `BufMut` are unable to defend against incorrect implementations
of `BufMut`, this makes the trait unsafe to implement.

Fixes #329
2020-10-16 15:45:38 -07:00
Carl LercheandGitHub 94c543f74b remove ext traits (#431) 2020-10-16 15:16:23 -07:00
Carl LercheandGitHub 447530b8a6 Remove BufMut::bytes_vectored_mut() (#430)
There are issues with regard to uninitialized memory. We are avoiding
stabilizing this function for now.
2020-10-16 11:56:03 -07:00
Carl LercheandGitHub 422048eb24 prepare for 0.6 work (#428) 2020-10-16 11:49:27 -07:00
Sean McArthur 6fdb7391ce v0.5.6 2020-07-13 17:58:36 -07:00
Sean McArthurandGitHub 025bec2954 Move loom to dev-dependencies (#416) 2020-07-13 16:55:47 -07:00
Taiki EndoandGitHub 2d51809bd7 Fix CI failure on Windows (#415)
--no-self-update is necessary because the windows environment cannot
self-update rustup.exe.
2020-07-13 12:33:30 +09:00
Taiki EndoandGitHub 7daa7fe053 Change default lint level to warning and deny warnings in CI (#397) 2020-07-09 09:12:39 -07:00
Sean LeatherandGitHub 972f538b7e docs: Clarify what BytesMut is (#375) 2020-07-09 09:11:45 -07:00
Bryan DonlanandSean McArthur 81550da474 BytesMut: Reuse buffer when data fully consumed via Buf
Closes #412
2020-07-08 15:28:36 -07:00
Tomasz MiąskoandGitHub 90e7e650c9 Use ThreadSanitizer CI setup that avoids false positives (#406) 2020-07-07 01:28:47 +09:00
Juan AguilarandGitHub bc4a6d56f4 Add inline attribute to BytesMut::as_mut (#410) 2020-07-05 17:43:26 -07:00
Juan AguilarandGitHub 3603cec7c2 Add inline attribute to BytesMut::set_len (#408) 2020-07-03 16:47:32 -07:00
Tomasz MiąskoandGitHub 5cde647c29 Remove unnecessary synchronization when cloning shared representation (#404) 2020-07-02 18:55:16 -07:00
Sean McArthur 008d3e508e v0.5.5 2020-06-18 14:30:11 -07:00
Mikhail ZabaluevandGitHub a3304e8b8b Remove outdated info on BytesMut::with_capacity (#394)
Remove a remark from the doc that pertained to the inline
layout and is no longer true.
2020-05-27 22:02:45 -07:00
Taiki EndoandGitHub 64fe7e10ab Migrate CI to GitHub Actions (#392) 2020-05-25 01:41:45 +09:00
Taiki EndoandGitHub e9877e7a3d Deny warnings for doc tests (#391) 2020-05-24 06:30:50 +09:00
Kirill FomichevandGitHub 08ec01d1e7 Fix reference in Take docs (#383) 2020-05-22 14:33:06 +09:00
Taiki EndoandGitHub 1fbf83816b Format with rustfmt (#389)
* Format with rustfmt

* Add rustfmt check to CI
2020-05-22 13:17:30 +09:00
Sean McArthurandGitHub 5e93fa4c6b Update to loom v0.3 (#381) 2020-05-22 11:23:22 +09:00
Cheng XUandGitHub 9f40abc12a Allow using serde feature in no_std environment (#385)
When `serde` is used, it would enable its default feature `std`.
This of course breaks no_std build.

Fix this by disabling serde's default features. This should work for
both serde + std and serde + no_std case.
2020-05-22 11:19:46 +09:00
Cheng XUandGitHub e8fa2511ff Fix tests for no_std build (#384)
This fixes `cargo test --no-default-features`.
2020-05-22 11:18:20 +09:00
Taiki EndoandGitHub fa464ca483 Fix CI failure on MacOS (#388)
macOS-10.13 has been removed.
2020-05-22 10:54:52 +09:00
Tim HambourgerandSean McArthur b4ebe8432e Fix #354 -- Make advance_mut impl of BufMut for Vec<u8> panic if cnt > remaining 2020-03-25 12:30:15 -07:00
Tim HambourgerandSean McArthur 8bbe9dd87b Fix #352 -- Make freeze respect the start offset for BytesMuts in Vec mode 2020-03-24 11:14:16 -07:00
Sean McArthur fe6e673864 v0.5.4 2020-01-23 10:41:50 -08:00
Stepan KoltsovandSean McArthur f330ef6c4d Do not panic on Bytes::slice_ref on empty slice (#355)
Use case:

```
let bytes: Bytes = ...
let subbytes = bytes.slice(a..b); // where a == b
let slice = &subbytes[..];
let slice_bytes = bytes.slice_ref(slice);
```

Last line should not panic, because `slice` object is derived from
the original `Bytes` object.

Before this commit it panics, because `Bytes::slice` returns a fresh
`Bytes` object when `begin == end`.
2020-01-23 10:29:42 -08:00
Sean McArthurandGitHub 87b75cce16 Disable loom on Windows (#366) 2020-01-23 10:10:15 -08:00
南浦月andSean McArthur 3853a1fac4 merge hex & debug into fmt (#357) 2020-01-23 10:08:40 -08:00
Nikhil BeneschandSean McArthur f9ebf74091 Minor documentation fixes (#351)
* Remove incorrect comment about behavior of put_slice
* Improve grammar and clarity of BytesMut documentation
2020-01-23 10:06:49 -08:00
Andrew Tunnell-JonesandSean McArthur 939a5edf3d Fix reversed arguments in PartialOrd impls (#358) 2020-01-23 10:06:00 -08:00
John-John TedroandSean McArthur 788cb158ce Assert that the alignment of Shared is appropriate (#362) 2020-01-23 10:05:20 -08:00
Stepan KoltsovandSean McArthur ab028eb6a8 rebuild_boxed_slice instead of rebuild_boxed_vec (#364)
"Promotable" `Bytes` object is constructed from disassembling a
boxed slice object, not a vec.

Thus we should reassemble data into a boxed slice, not into a vec.

Although, it does not create any problems in practice (`Box<[u8]>`
is allocated exactly the same way as `Vec<u8>`), technically it is
a violation of `Vec::from_raw_parts` spec which says that a pointer
"needs to have been previously allocated via `String`/`Vec<T>`".
2020-01-23 10:04:13 -08:00
Sean McArthurandCarl Lerche e0eebde993 Fix Bytes::truncate losing the original Vec's capacity (#361) 2020-01-22 16:37:53 -08:00
Stepan KoltsovandSean McArthur 729bc7c208 Make Bytes::new const fn (#356) 2020-01-09 14:27:28 -08:00
Sean McArthurandCarl Lerche 8695c08bcc Improve assertion messages (#349) 2020-01-07 21:29:36 -08:00
Thayne McCombsandSean McArthur 39b6646e66 Re-add impl From<BytesMut> for Bytes (#348) 2019-12-19 11:09:14 -08:00
Sean McArthurandGitHub c7cf716180 Fix Bytes when Vec pointer's LSB is set (#346)
This separates the `SharedVtable` into 3:

- `PromotableEvenVtable`: The original `SharedVtable`, which will
  promote the `Vec` to `Shared` on the first clone, and is selected when
  the `Vec`'s pointer has the LSB unset.
- `PromotableOddVtable`: Similar to the `PromotableEvenVtable`, but
  selected when the `Vec`'s pointer has the LSB set. This vtable differs
  in the masking used when reconstructing the `Vec`.
- `SharedVtable`: This no longer checks if its current kind is `VEC` or
  `ARC`, and is only created by the "promotable" vtables.

This also adds a test using an "odd" global allocator that purposefully
bumps all pointers with alignment of 1.

Closes #343
2019-12-17 13:23:15 -08:00
Sean McArthur 8ae3bb2104 v0.5.3 2019-12-12 12:00:21 -08:00
Sean McArthur 8733f74d59 Assert the LSB is 0 when converting Vec into Bytes 2019-12-12 11:46:51 -08:00
Steven FacklerandSean McArthur 17a8ac91e0 Fix conversion of empty vectors to Bytes
Closes #340
2019-12-12 08:33:07 -08:00
Sean McArthur a7fc5274ad Make Deref impls of Buf and BufMut forward more methods 2019-12-10 13:35:26 -08:00
Sean McArthurandCarl Lerche 7e80f3b646 Optimize BufMut for Vec and BytesMut (#338) 2019-12-10 13:30:49 -08:00
Sean McArthur a4908213a6 Add must_use to split, split_off, and split_to 2019-12-04 12:39:48 -08:00
Mikhail ZabaluevandCarl Lerche af606aab9b Fix regression in Bytes::truncate (#333)
When the length to truncate is greater than the buffer's current
length, do nothing instead of clearing the contents.
2019-12-01 14:00:42 -08:00
50 changed files with 2426 additions and 1243 deletions
+172
View File
@@ -0,0 +1,172 @@
name: CI
on:
pull_request:
branches:
- master
push:
branches:
- master
env:
RUSTFLAGS: -Dwarnings
RUST_BACKTRACE: 1
nightly: nightly-2021-04-13
defaults:
run:
shell: bash
jobs:
# Check formatting
rustfmt:
name: rustfmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable && rustup default stable
- name: Check formatting
run: cargo fmt --all -- --check
# TODO
# # Apply clippy lints
# clippy:
# name: clippy
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v2
# - name: Apply clippy lints
# run: cargo clippy --all-features
# This represents the minimum Rust version supported by
# Bytes. Updating this should be done in a dedicated PR.
#
# Tests are not run as tests may require newer versions of
# rust.
minrust:
name: minrust
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update 1.39.0 && rustup default 1.39.0
- name: Check
run: . ci/test-stable.sh check
# Stable
stable:
name: stable
strategy:
matrix:
os:
- ubuntu-latest
- macos-latest
- windows-latest
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- name: Install Rust
# --no-self-update is necessary because the windows environment cannot self-update rustup.exe.
run: rustup update stable --no-self-update && rustup default stable
- name: Test
run: . ci/test-stable.sh test
# Nightly
nightly:
name: nightly
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update $nightly && rustup default $nightly
- name: Test
run: . ci/test-stable.sh test
# Run tests on some extra platforms
cross:
name: cross
strategy:
matrix:
target:
- i686-unknown-linux-gnu
- armv7-unknown-linux-gnueabihf
- powerpc-unknown-linux-gnu
- powerpc64-unknown-linux-gnu
- wasm32-unknown-unknown
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable && rustup default stable
- name: cross build --target ${{ matrix.target }}
run: |
cargo install cross
cross build --target ${{ matrix.target }}
if: matrix.target != 'wasm32-unknown-unknown'
# WASM support
- name: cargo build --target ${{ matrix.target }}
run: |
rustup target add ${{ matrix.target }}
cargo build --target ${{ matrix.target }}
if: matrix.target == 'wasm32-unknown-unknown'
# Sanitizers
tsan:
name: tsan
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update $nightly && rustup default $nightly
- name: Install rust-src
run: rustup component add rust-src
- name: ASAN / TSAN
run: . ci/tsan.sh
miri:
name: miri
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Miri
run: ci/miri.sh
# Loom
loom:
name: loom
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update $nightly && rustup default $nightly
- name: Loom tests
run: RUSTFLAGS="--cfg loom -Dwarnings" cargo test --lib
publish_docs:
name: Publish Documentation
needs:
- rustfmt
# - clippy
- stable
- nightly
- minrust
- cross
- tsan
- loom
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Rust
run: rustup update stable && rustup default stable
- name: Build documentation
run: cargo doc --no-deps --all-features
env:
RUSTDOCFLAGS: --cfg docsrs
- name: Publish documentation
run: |
cd target/doc
git init
git add .
git -c user.name='ci' -c user.email='ci' commit -m 'Deploy Bytes API documentation'
git push -f -q https://git:${{ secrets.github_token }}@github.com/${{ github.repository }} HEAD:gh-pages
if: github.event_name == 'push' && github.event.ref == 'refs/heads/master' && github.repository == 'tokio-rs/bytes'
+91
View File
@@ -1,3 +1,94 @@
# 1.1.0 (August 25, 2021)
### Added
- `BufMut::put_bytes(self, val, cnt)` (#487)
- Implement `From<Box<[u8]>>` for `Bytes` (#504)
### Changed
- Override `put_slice` for `&mut [u8]` (#483)
- Panic on integer overflow in `Chain::remaining` (#482)
- Add inline tags to `UninitSlice` methods (#443)
- Override `copy_to_bytes` for Chain and Take (#481)
- Keep capacity when unsplit on empty other buf (#502)
### Documented
- Clarify `BufMut` allocation guarantees (#501)
- Clarify `BufMut::put_int` behavior (#486)
- Clarify actions of `clear` and `truncate`. (#508)
# 1.0.1 (January 11, 2021)
### Changed
- mark `Vec::put_slice` with `#[inline]` (#459)
### Fixed
- Fix deprecation warning (#457)
- use `Box::into_raw` instead of `mem::forget`-in-disguise (#458)
# 1.0.0 (December 22, 2020)
### Changed
- Rename `Buf`/`BufMut` methods `bytes()` and `bytes_mut()` to `chunk()` and `chunk_mut()` (#450)
### Removed
- remove unused Buf implementation. (#449)
# 0.6.0 (October 21, 2020)
API polish in preparation for a 1.0 release.
### Changed
- `BufMut` is now an `unsafe` trait (#432).
- `BufMut::bytes_mut()` returns `&mut UninitSlice`, a type owned by `bytes` to
avoid undefined behavior (#433).
- `Buf::copy_to_bytes(len)` replaces `Buf::into_bytes()` (#439).
- `Buf`/`BufMut` utility methods are moved onto the trait and `*Ext` traits are
removed (#431).
### Removed
- `BufMut::bytes_vectored_mut()` (#430).
- `new` methods on combinator types (#434).
# 0.5.6 (July 13, 2020)
- Improve `BytesMut` to reuse buffer when fully `advance`d.
- Mark `BytesMut::{as_mut, set_len}` with `#[inline]`.
- Relax synchronization when cloning in shared vtable of `Bytes`.
- Move `loom` to `dev-dependencies`.
# 0.5.5 (June 18, 2020)
### Added
- Allow using the `serde` feature in `no_std` environments (#385).
### Fix
- Fix `BufMut::advance_mut` to panic if advanced passed the capacity (#354)..
- Fix `BytesMut::freeze` ignoring amount previously `advance`d (#352).
# 0.5.4 (January 23, 2020)
### Added
- Make `Bytes::new` a `const fn`.
- Add `From<BytesMut>` for `Bytes`.
### Fix
- Fix reversed arguments in `PartialOrd` for `Bytes`.
- Fix `Bytes::truncate` losing original capacity when repr is an unshared `Vec`.
- Fix `Bytes::from(Vec)` when allocator gave `Vec` a pointer with LSB set.
- Fix panic in `Bytes::slice_ref` if argument is an empty slice.
# 0.5.3 (December 12, 2019)
### Added
- `must_use` attributes to `split`, `split_off`, and `split_to` methods (#337).
### Fix
- Potential freeing of a null pointer in `Bytes` when constructed with an empty `Vec<u8>` (#341, #342).
- Calling `Bytes::truncate` with a size large than the length will no longer clear the `Bytes` (#333).
# 0.5.2 (November 27, 2019)
### Added
+13 -8
View File
@@ -2,15 +2,15 @@
name = "bytes"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Update doc URL.
# - Create "v0.5.x" git tag.
version = "0.5.2"
# - Create "v1.x.y" git tag.
version = "1.1.0"
license = "MIT"
authors = ["Carl Lerche <[email protected]>"]
authors = [
"Carl Lerche <[email protected]>",
"Sean McArthur <[email protected]>",
]
description = "Types and traits for working with bytes"
documentation = "https://docs.rs/bytes"
repository = "https://github.com/tokio-rs/bytes"
readme = "README.md"
keywords = ["buffers", "zero-copy", "io"]
@@ -22,8 +22,13 @@ default = ["std"]
std = []
[dependencies]
serde = { version = "1.0", optional = true }
serde = { version = "1.0.60", optional = true, default-features = false, features = ["alloc"] }
[dev-dependencies]
loom = "0.2.10"
serde_test = "1.0"
[target.'cfg(loom)'.dev-dependencies]
loom = "0.5"
[package.metadata.docs.rs]
rustdoc-args = ["--cfg", "docsrs"]
+5 -6
View File
@@ -3,12 +3,12 @@
A utility library for working with bytes.
[![Crates.io][crates-badge]][crates-url]
[![Build Status][azure-badge]][azure-url]
[![Build Status][ci-badge]][ci-url]
[crates-badge]: https://img.shields.io/crates/v/bytes.svg
[crates-url]: https://crates.io/crates/bytes
[azure-badge]: https://dev.azure.com/tokio-rs/bytes/_apis/build/status/tokio-rs.bytes?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/bytes/_build/latest?definitionId=3&branchName=master
[ci-badge]: https://github.com/tokio-rs/bytes/workflows/CI/badge.svg
[ci-url]: https://github.com/tokio-rs/bytes/actions
[Documentation](https://docs.rs/bytes)
@@ -18,7 +18,7 @@ To use `bytes`, first add this to your `Cargo.toml`:
```toml
[dependencies]
bytes = "0.5"
bytes = "1"
```
Next, add this to your crate:
@@ -33,7 +33,7 @@ Serde support is optional and disabled by default. To enable use the feature `se
```toml
[dependencies]
bytes = { version = "0.5", features = ["serde"] }
bytes = { version = "1", features = ["serde"] }
```
## License
@@ -45,4 +45,3 @@ This project is licensed under the [MIT license](LICENSE).
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in `bytes` by you, shall be licensed as MIT, without any additional
terms or conditions.
-68
View File
@@ -1,68 +0,0 @@
trigger: ["master"]
pr: ["master"]
jobs:
# Check formatting
# - template: ci/azure-rustfmt.yml
# parameters:
# name: rustfmt
# Apply clippy lints
# - template: ci/azure-clippy.yml
# parameters:
# name: clippy
# This represents the minimum Rust version supported by
# Bytes. Updating this should be done in a dedicated PR.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-test-stable.yml
parameters:
name: minrust
rust_version: 1.39.0
cmd: check
# Stable
- template: ci/azure-test-stable.yml
parameters:
name: stable
cross: true
features:
- serde
# Nightly
- template: ci/azure-test-stable.yml
parameters:
name: nightly
# Pin nightly to avoid being impacted by breakage
rust_version: nightly-2019-09-25
benches: true
# Run tests on some extra platforms
- template: ci/azure-cross-compile.yml
parameters:
name: cross
# Sanitizers
- template: ci/azure-tsan.yml
parameters:
name: tsan
rust_version: nightly
# Loom
- template: ci/azure-loom.yml
parameters:
name: loom
rust_version: stable
- template: ci/azure-deploy-docs.yml
parameters:
dependsOn:
# - rustfmt
# - clippy
- stable
- nightly
- minrust
- cross
+5 -6
View File
@@ -1,10 +1,10 @@
#![feature(test)]
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
extern crate test;
use test::Bencher;
use bytes::Buf;
use test::Bencher;
/// Dummy Buf implementation
struct TestBuf {
@@ -53,7 +53,7 @@ impl Buf for TestBuf {
assert!(self.pos <= self.buf.len());
self.next_readlen();
}
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
if self.readlen == 0 {
Default::default()
} else {
@@ -87,8 +87,8 @@ impl Buf for TestBufC {
self.inner.advance(cnt)
}
#[inline(never)]
fn bytes(&self) -> &[u8] {
self.inner.bytes()
fn chunk(&self) -> &[u8] {
self.inner.chunk()
}
}
@@ -159,7 +159,6 @@ macro_rules! bench_group {
mod get_u8 {
use super::*;
bench_group!(get_u8);
bench!(option, option);
}
mod get_u16 {
use super::*;
+4 -3
View File
@@ -1,10 +1,10 @@
#![feature(test)]
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
extern crate test;
use test::Bencher;
use bytes::Bytes;
use test::Bencher;
#[bench]
fn deref_unique(b: &mut Bencher) {
@@ -42,7 +42,8 @@ fn deref_static(b: &mut Bencher) {
#[bench]
fn clone_static(b: &mut Bencher) {
let bytes = Bytes::from_static("hello world 1234567890 and have a good byte 0987654321".as_bytes());
let bytes =
Bytes::from_static("hello world 1234567890 and have a good byte 0987654321".as_bytes());
b.iter(|| {
for _ in 0..1024 {
+29 -12
View File
@@ -1,10 +1,10 @@
#![feature(test)]
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
extern crate test;
use test::Bencher;
use bytes::{BufMut, BytesMut};
use test::Bencher;
#[bench]
fn alloc_small(b: &mut Bencher) {
@@ -29,7 +29,6 @@ fn alloc_big(b: &mut Bencher) {
})
}
#[bench]
fn deref_unique(b: &mut Bencher) {
let mut buf = BytesMut::with_capacity(4096);
@@ -92,7 +91,9 @@ fn deref_two(b: &mut Bencher) {
#[bench]
fn clone_frozen(b: &mut Bencher) {
let bytes = BytesMut::from(&b"hello world 1234567890 and have a good byte 0987654321"[..]).split().freeze();
let bytes = BytesMut::from(&b"hello world 1234567890 and have a good byte 0987654321"[..])
.split()
.freeze();
b.iter(|| {
for _ in 0..1024 {
@@ -137,7 +138,9 @@ fn fmt_write(b: &mut Bencher) {
b.iter(|| {
let _ = write!(buf, "{}", s);
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
})
}
@@ -152,7 +155,9 @@ fn bytes_mut_extend(b: &mut Bencher) {
buf.extend(&data);
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
@@ -169,7 +174,9 @@ fn put_slice_bytes_mut(b: &mut Bencher) {
buf.put_slice(&data);
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
@@ -184,7 +191,9 @@ fn put_u8_bytes_mut(b: &mut Bencher) {
buf.put_u8(b'x');
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
@@ -199,7 +208,9 @@ fn put_slice_vec(b: &mut Bencher) {
buf.put_slice(&data);
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
@@ -214,7 +225,9 @@ fn put_u8_vec(b: &mut Bencher) {
buf.put_u8(b'x');
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
@@ -229,7 +242,9 @@ fn put_slice_vec_extend(b: &mut Bencher) {
buf.extend_from_slice(&data);
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
@@ -244,6 +259,8 @@ fn put_u8_vec_push(b: &mut Bencher) {
buf.push(b'x');
}
test::black_box(&buf);
unsafe { buf.set_len(0); }
unsafe {
buf.set_len(0);
}
});
}
-46
View File
@@ -1,46 +0,0 @@
parameters:
cmd: build
rust_version: stable
jobs:
- job: ${{ parameters.name }}
displayName: Cross
strategy:
matrix:
i686:
vmImage: ubuntu-16.04
target: i686-unknown-linux-gnu
armv7:
vmImage: ubuntu-16.04
target: armv7-unknown-linux-gnueabihf
powerpc:
vmImage: ubuntu-16.04
target: powerpc-unknown-linux-gnu
powerpc64:
vmImage: ubuntu-16.04
target: powerpc64-unknown-linux-gnu
wasm:
vmImage: ubuntu-16.04
target: wasm32-unknown-unknown
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{parameters.rust_version}}
- script: cargo install cross
displayName: Install cross
condition: not(eq(variables['target'], 'wasm32-unknown-unknown'))
- script: cross ${{ parameters.cmd }} --target $(target)
displayName: cross ${{ parameters.cmd }} --target $(target)
condition: not(eq(variables['target'], 'wasm32-unknown-unknown'))
# WASM support
- script: |
rustup target add $(target)
cargo build --target $(target)
displayName: cargo build --target $(target)
condition: eq(variables['target'], 'wasm32-unknown-unknown')
-39
View File
@@ -1,39 +0,0 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
cargo doc --no-deps
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/bytes
git checkout -b gh-pages
git add .
git commit -m 'Deploy Bytes API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
-33
View File
@@ -1,33 +0,0 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain none
export PATH=$PATH:$HOME/.cargo/bin
rustup toolchain install $RUSTUP_TOOLCHAIN
rustup default $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain none
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
rustup toolchain install %RUSTUP_TOOLCHAIN%
rustup default %RUSTUP_TOOLCHAIN%
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustup toolchain list
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
-15
View File
@@ -1,15 +0,0 @@
jobs:
- job: ${{parameters.name}}
displayName: Loom tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{parameters.rust_version}}
- script: RUSTFLAGS="--cfg loom" cargo test --lib
displayName: RUSTFLAGS="--cfg loom" cargo test --lib
-50
View File
@@ -1,50 +0,0 @@
parameters:
cmd: test
rust_version: stable
features: []
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{parameters.rust_version}}
# Run with default crate features
- script: cargo ${{ parameters.cmd }}
displayName: cargo ${{ parameters.cmd }}
# Run with each specified feature
- ${{ each feature in parameters.features }}:
- script: cargo ${{ parameters.cmd }} --features ${{ feature }}
displayName: cargo ${{ parameters.cmd }} --features ${{ feature }}
- ${{ if eq(parameters.cmd, 'test') }}:
- script: cargo doc --no-deps
displayName: cargo doc --no-deps
- ${{ if parameters.benches }}:
- script: cargo check --benches
displayName: Check benchmarks
# Run with all features
- script: cargo ${{ parameters.cmd }} --all-features
displayName: cargo ${{ parameters.cmd }} --all-features
# Run with no default features
- script: cargo check --no-default-features
displayName: cargo check --no-default-features
-26
View File
@@ -1,26 +0,0 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- script: |
set -e
export RUST_TEST_THREADS=1
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test --target x86_64-unknown-linux-gnu --test test_bytes --test test_buf --test test_buf_mut
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test --target x86_64-unknown-linux-gnu --test test_bytes --test test_buf --test test_buf_mut
displayName: TSAN / MSAN
Executable
+11
View File
@@ -0,0 +1,11 @@
#!/bin/bash
set -e
MIRI_NIGHTLY=nightly-$(curl -s https://rust-lang.github.io/rustup-components-history/x86_64-unknown-linux-gnu/miri)
echo "Installing latest nightly with Miri: $MIRI_NIGHTLY"
rustup set profile minimal
rustup default "$MIRI_NIGHTLY"
rustup component add miri
cargo miri test
cargo miri test --target mips64-unknown-linux-gnuabi64
+28
View File
@@ -0,0 +1,28 @@
#!/bin/bash
set -ex
cmd="${1:-test}"
# Install cargo-hack for feature flag test
host=$(rustc -Vv | grep host | sed 's/host: //')
curl -LsSf https://github.com/taiki-e/cargo-hack/releases/latest/download/cargo-hack-$host.tar.gz | tar xzf - -C ~/.cargo/bin
# Run with each feature
# * --each-feature includes both default/no-default features
# * --optional-deps is needed for serde feature
cargo hack "${cmd}" --each-feature --optional-deps
# Run with all features
cargo "${cmd}" --all-features
cargo doc --no-deps --all-features
if [[ "${RUST_VERSION}" == "nightly"* ]]; then
# Check benchmarks
cargo check --benches
# Check minimal versions
cargo clean
cargo update -Zminimal-versions
cargo check --all-features
fi
-24
View File
@@ -1,24 +0,0 @@
# TSAN suppressions file for `bytes`
# TSAN does not understand fences and `Arc::drop` is implemented using a fence.
# This causes many false positives.
race:Arc*drop
race:arc*Weak*drop
# `std` mpsc is not used in any Bytes code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
# Some test runtime races. Allocation should be race free
race:alloc::alloc
# Not sure why this is warning, but it is in the test harness and not the library.
race:TestEvent*clone
race:test::run_tests_console::*closure
# Probably more fences in std.
race:__call_tls_dtors
# This ignores a false positive caused by `thread::park()`/`thread::unpark()`.
# See: https://github.com/rust-lang/rust/pull/54806#issuecomment-436193353
race:pthread_cond_destroy
+13
View File
@@ -0,0 +1,13 @@
#!/bin/bash
set -ex
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test --target x86_64-unknown-linux-gnu --test test_bytes --test test_buf --test test_buf_mut
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo -Zbuild-std test --target x86_64-unknown-linux-gnu --test test_bytes --test test_buf --test test_buf_mut
+245 -101
View File
@@ -1,22 +1,27 @@
use core::{cmp, ptr, mem};
#[cfg(feature = "std")]
use crate::buf::{reader, Reader};
use crate::buf::{take, Chain, Take};
use core::{cmp, mem, ptr};
#[cfg(feature = "std")]
use std::io::IoSlice;
use alloc::{boxed::Box};
use alloc::boxed::Box;
macro_rules! buf_get_impl {
($this:ident, $typ:tt::$conv:tt) => ({
($this:ident, $typ:tt::$conv:tt) => {{
const SIZE: usize = mem::size_of::<$typ>();
// try to convert directly from the bytes
// this Option<ret> trick is to avoid keeping a borrow on self
// when advance() is called (mut borrow) and to call bytes() only once
let ret = $this.bytes().get(..SIZE).map(|src| unsafe {
$typ::$conv(*(src as *const _ as *const [_; SIZE]))
});
// try to convert directly from the bytes
// this Option<ret> trick is to avoid keeping a borrow on self
// when advance() is called (mut borrow) and to call bytes() only once
let ret = $this
.chunk()
.get(..SIZE)
.map(|src| unsafe { $typ::$conv(*(src as *const _ as *const [_; SIZE])) });
if let Some(ret) = ret {
// if the direct conversion was possible, advance and return
// if the direct conversion was possible, advance and return
$this.advance(SIZE);
return ret;
} else {
@@ -25,8 +30,8 @@ macro_rules! buf_get_impl {
$this.copy_to_slice(&mut buf); // (do the advance)
return $typ::$conv(buf);
}
});
(le => $this:ident, $typ:tt, $len_to_read:expr) => ({
}};
(le => $this:ident, $typ:tt, $len_to_read:expr) => {{
debug_assert!(mem::size_of::<$typ>() >= $len_to_read);
// The same trick as above does not improve the best case speed.
@@ -34,12 +39,12 @@ macro_rules! buf_get_impl {
let mut buf = [0; (mem::size_of::<$typ>())];
$this.copy_to_slice(&mut buf[..($len_to_read)]);
return $typ::from_le_bytes(buf);
});
}};
(be => $this:ident, $typ:tt, $len_to_read:expr) => {{
debug_assert!(mem::size_of::<$typ>() >= $len_to_read);
let mut buf = [0; (mem::size_of::<$typ>())];
$this.copy_to_slice(&mut buf[mem::size_of::<$typ>()-($len_to_read)..]);
$this.copy_to_slice(&mut buf[mem::size_of::<$typ>() - ($len_to_read)..]);
return $typ::from_be_bytes(buf);
}};
}
@@ -73,7 +78,7 @@ pub trait Buf {
/// the buffer.
///
/// This value is greater than or equal to the length of the slice returned
/// by `bytes`.
/// by `chunk()`.
///
/// # Examples
///
@@ -110,31 +115,34 @@ pub trait Buf {
///
/// let mut buf = &b"hello world"[..];
///
/// assert_eq!(buf.bytes(), &b"hello world"[..]);
/// assert_eq!(buf.chunk(), &b"hello world"[..]);
///
/// buf.advance(6);
///
/// assert_eq!(buf.bytes(), &b"world"[..]);
/// assert_eq!(buf.chunk(), &b"world"[..]);
/// ```
///
/// # Implementer notes
///
/// This function should never panic. Once the end of the buffer is reached,
/// i.e., `Buf::remaining` returns 0, calls to `bytes` should return an
/// i.e., `Buf::remaining` returns 0, calls to `chunk()` should return an
/// empty slice.
fn bytes(&self) -> &[u8];
// The `chunk` method was previously called `bytes`. This alias makes the rename
// more easily discoverable.
#[cfg_attr(docsrs, doc(alias = "bytes"))]
fn chunk(&self) -> &[u8];
/// Fills `dst` with potentially multiple slices starting at `self`'s
/// current position.
///
/// If the `Buf` is backed by disjoint slices of bytes, `bytes_vectored` enables
/// If the `Buf` is backed by disjoint slices of bytes, `chunk_vectored` enables
/// fetching more than one slice at once. `dst` is a slice of `IoSlice`
/// references, enabling the slice to be directly used with [`writev`]
/// without any further conversion. The sum of the lengths of all the
/// buffers in `dst` will be less than or equal to `Buf::remaining()`.
///
/// The entries in `dst` will be overwritten, but the data **contained** by
/// the slices **will not** be modified. If `bytes_vectored` does not fill every
/// the slices **will not** be modified. If `chunk_vectored` does not fill every
/// entry in `dst`, then `dst` is guaranteed to contain all remaining slices
/// in `self.
///
@@ -144,7 +152,7 @@ pub trait Buf {
/// # Implementer notes
///
/// This function should never panic. Once the end of the buffer is reached,
/// i.e., `Buf::remaining` returns 0, calls to `bytes_vectored` must return 0
/// i.e., `Buf::remaining` returns 0, calls to `chunk_vectored` must return 0
/// without mutating `dst`.
///
/// Implementations should also take care to properly handle being called
@@ -152,13 +160,13 @@ pub trait Buf {
///
/// [`writev`]: http://man7.org/linux/man-pages/man2/readv.2.html
#[cfg(feature = "std")]
fn bytes_vectored<'a>(&'a self, dst: &mut [IoSlice<'a>]) -> usize {
fn chunks_vectored<'a>(&'a self, dst: &mut [IoSlice<'a>]) -> usize {
if dst.is_empty() {
return 0;
}
if self.has_remaining() {
dst[0] = IoSlice::new(self.bytes());
dst[0] = IoSlice::new(self.chunk());
1
} else {
0
@@ -167,7 +175,7 @@ pub trait Buf {
/// Advance the internal cursor of the Buf
///
/// The next call to `bytes` will return a slice starting `cnt` bytes
/// The next call to `chunk()` will return a slice starting `cnt` bytes
/// further into the underlying buffer.
///
/// # Examples
@@ -177,11 +185,11 @@ pub trait Buf {
///
/// let mut buf = &b"hello world"[..];
///
/// assert_eq!(buf.bytes(), &b"hello world"[..]);
/// assert_eq!(buf.chunk(), &b"hello world"[..]);
///
/// buf.advance(6);
///
/// assert_eq!(buf.bytes(), &b"world"[..]);
/// assert_eq!(buf.chunk(), &b"world"[..]);
/// ```
///
/// # Panics
@@ -248,11 +256,10 @@ pub trait Buf {
let cnt;
unsafe {
let src = self.bytes();
let src = self.chunk();
cnt = cmp::min(src.len(), dst.len() - off);
ptr::copy_nonoverlapping(
src.as_ptr(), dst[off..].as_mut_ptr(), cnt);
ptr::copy_nonoverlapping(src.as_ptr(), dst[off..].as_mut_ptr(), cnt);
off += cnt;
}
@@ -279,7 +286,7 @@ pub trait Buf {
/// This function panics if there is no more remaining data in `self`.
fn get_u8(&mut self) -> u8 {
assert!(self.remaining() >= 1);
let ret = self.bytes()[0];
let ret = self.chunk()[0];
self.advance(1);
ret
}
@@ -302,7 +309,7 @@ pub trait Buf {
/// This function panics if there is no more remaining data in `self`.
fn get_i8(&mut self) -> i8 {
assert!(self.remaining() >= 1);
let ret = self.bytes()[0] as i8;
let ret = self.chunk()[0] as i8;
self.advance(1);
ret
}
@@ -791,60 +798,224 @@ pub trait Buf {
f64::from_bits(Self::get_u64_le(self))
}
/// Consumes remaining bytes inside self and returns new instance of `Bytes`
/// Consumes `len` bytes inside self and returns new instance of `Bytes`
/// with this data.
///
/// This function may be optimized by the underlying type to avoid actual
/// copies. For example, `Bytes` implementation will do a shallow copy
/// (ref-count increment).
///
/// # Examples
///
/// ```
/// use bytes::Buf;
///
/// let bytes = (&b"hello world"[..]).to_bytes();
/// assert_eq!(&bytes[..], &b"hello world"[..]);
/// let bytes = (&b"hello world"[..]).copy_to_bytes(5);
/// assert_eq!(&bytes[..], &b"hello"[..]);
/// ```
fn to_bytes(&mut self) -> crate::Bytes {
fn copy_to_bytes(&mut self, len: usize) -> crate::Bytes {
use super::BufMut;
let mut ret = crate::BytesMut::with_capacity(self.remaining());
ret.put(self);
assert!(len <= self.remaining(), "`len` greater than remaining");
let mut ret = crate::BytesMut::with_capacity(len);
ret.put(self.take(len));
ret.freeze()
}
/// Creates an adaptor which will read at most `limit` bytes from `self`.
///
/// This function returns a new instance of `Buf` which will read at most
/// `limit` bytes.
///
/// # Examples
///
/// ```
/// use bytes::{Buf, BufMut};
///
/// let mut buf = b"hello world"[..].take(5);
/// let mut dst = vec![];
///
/// dst.put(&mut buf);
/// assert_eq!(dst, b"hello");
///
/// let mut buf = buf.into_inner();
/// dst.clear();
/// dst.put(&mut buf);
/// assert_eq!(dst, b" world");
/// ```
fn take(self, limit: usize) -> Take<Self>
where
Self: Sized,
{
take::new(self, limit)
}
/// Creates an adaptor which will chain this buffer with another.
///
/// The returned `Buf` instance will first consume all bytes from `self`.
/// Afterwards the output is equivalent to the output of next.
///
/// # Examples
///
/// ```
/// use bytes::Buf;
///
/// let mut chain = b"hello "[..].chain(&b"world"[..]);
///
/// let full = chain.copy_to_bytes(11);
/// assert_eq!(full.chunk(), b"hello world");
/// ```
fn chain<U: Buf>(self, next: U) -> Chain<Self, U>
where
Self: Sized,
{
Chain::new(self, next)
}
/// Creates an adaptor which implements the `Read` trait for `self`.
///
/// This function returns a new value which implements `Read` by adapting
/// the `Read` trait functions to the `Buf` trait functions. Given that
/// `Buf` operations are infallible, none of the `Read` functions will
/// return with `Err`.
///
/// # Examples
///
/// ```
/// use bytes::{Bytes, Buf};
/// use std::io::Read;
///
/// let buf = Bytes::from("hello world");
///
/// let mut reader = buf.reader();
/// let mut dst = [0; 1024];
///
/// let num = reader.read(&mut dst).unwrap();
///
/// assert_eq!(11, num);
/// assert_eq!(&dst[..11], &b"hello world"[..]);
/// ```
#[cfg(feature = "std")]
fn reader(self) -> Reader<Self>
where
Self: Sized,
{
reader::new(self)
}
}
macro_rules! deref_forward_buf {
() => {
fn remaining(&self) -> usize {
(**self).remaining()
}
fn chunk(&self) -> &[u8] {
(**self).chunk()
}
#[cfg(feature = "std")]
fn chunks_vectored<'b>(&'b self, dst: &mut [IoSlice<'b>]) -> usize {
(**self).chunks_vectored(dst)
}
fn advance(&mut self, cnt: usize) {
(**self).advance(cnt)
}
fn has_remaining(&self) -> bool {
(**self).has_remaining()
}
fn copy_to_slice(&mut self, dst: &mut [u8]) {
(**self).copy_to_slice(dst)
}
fn get_u8(&mut self) -> u8 {
(**self).get_u8()
}
fn get_i8(&mut self) -> i8 {
(**self).get_i8()
}
fn get_u16(&mut self) -> u16 {
(**self).get_u16()
}
fn get_u16_le(&mut self) -> u16 {
(**self).get_u16_le()
}
fn get_i16(&mut self) -> i16 {
(**self).get_i16()
}
fn get_i16_le(&mut self) -> i16 {
(**self).get_i16_le()
}
fn get_u32(&mut self) -> u32 {
(**self).get_u32()
}
fn get_u32_le(&mut self) -> u32 {
(**self).get_u32_le()
}
fn get_i32(&mut self) -> i32 {
(**self).get_i32()
}
fn get_i32_le(&mut self) -> i32 {
(**self).get_i32_le()
}
fn get_u64(&mut self) -> u64 {
(**self).get_u64()
}
fn get_u64_le(&mut self) -> u64 {
(**self).get_u64_le()
}
fn get_i64(&mut self) -> i64 {
(**self).get_i64()
}
fn get_i64_le(&mut self) -> i64 {
(**self).get_i64_le()
}
fn get_uint(&mut self, nbytes: usize) -> u64 {
(**self).get_uint(nbytes)
}
fn get_uint_le(&mut self, nbytes: usize) -> u64 {
(**self).get_uint_le(nbytes)
}
fn get_int(&mut self, nbytes: usize) -> i64 {
(**self).get_int(nbytes)
}
fn get_int_le(&mut self, nbytes: usize) -> i64 {
(**self).get_int_le(nbytes)
}
fn copy_to_bytes(&mut self, len: usize) -> crate::Bytes {
(**self).copy_to_bytes(len)
}
};
}
impl<T: Buf + ?Sized> Buf for &mut T {
fn remaining(&self) -> usize {
(**self).remaining()
}
fn bytes(&self) -> &[u8] {
(**self).bytes()
}
#[cfg(feature = "std")]
fn bytes_vectored<'b>(&'b self, dst: &mut [IoSlice<'b>]) -> usize {
(**self).bytes_vectored(dst)
}
fn advance(&mut self, cnt: usize) {
(**self).advance(cnt)
}
deref_forward_buf!();
}
impl<T: Buf + ?Sized> Buf for Box<T> {
fn remaining(&self) -> usize {
(**self).remaining()
}
fn bytes(&self) -> &[u8] {
(**self).bytes()
}
#[cfg(feature = "std")]
fn bytes_vectored<'b>(&'b self, dst: &mut [IoSlice<'b>]) -> usize {
(**self).bytes_vectored(dst)
}
fn advance(&mut self, cnt: usize) {
(**self).advance(cnt)
}
deref_forward_buf!();
}
impl Buf for &[u8] {
@@ -854,7 +1025,7 @@ impl Buf for &[u8] {
}
#[inline]
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
self
}
@@ -864,34 +1035,6 @@ impl Buf for &[u8] {
}
}
impl Buf for Option<[u8; 1]> {
fn remaining(&self) -> usize {
if self.is_some() {
1
} else {
0
}
}
fn bytes(&self) -> &[u8] {
self.as_ref().map(AsRef::as_ref)
.unwrap_or(Default::default())
}
fn advance(&mut self, cnt: usize) {
if cnt == 0 {
return;
}
if self.is_none() {
panic!("overflow");
} else {
assert_eq!(1, cnt);
*self = None;
}
}
}
#[cfg(feature = "std")]
impl<T: AsRef<[u8]>> Buf for std::io::Cursor<T> {
fn remaining(&self) -> usize {
@@ -905,7 +1048,7 @@ impl<T: AsRef<[u8]>> Buf for std::io::Cursor<T> {
len - pos as usize
}
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
let len = self.get_ref().as_ref().len();
let pos = self.position();
@@ -918,7 +1061,8 @@ impl<T: AsRef<[u8]>> Buf for std::io::Cursor<T> {
fn advance(&mut self, cnt: usize) {
let pos = (self.position() as usize)
.checked_add(cnt).expect("overflow");
.checked_add(cnt)
.expect("overflow");
assert!(pos <= self.get_ref().as_ref().len());
self.set_position(pos as u64);
+305 -175
View File
@@ -1,9 +1,10 @@
use core::{cmp, mem::{self, MaybeUninit}, ptr, usize};
use crate::buf::{limit, Chain, Limit, UninitSlice};
#[cfg(feature = "std")]
use std::fmt;
use crate::buf::{writer, Writer};
use alloc::{vec::Vec, boxed::Box};
use core::{cmp, mem, ptr, usize};
use alloc::{boxed::Box, vec::Vec};
/// A trait for values that provide sequential write access to bytes.
///
@@ -25,12 +26,16 @@ use alloc::{vec::Vec, boxed::Box};
///
/// assert_eq!(buf, b"hello world");
/// ```
pub trait BufMut {
pub unsafe trait BufMut {
/// Returns the number of bytes that can be written from the current
/// position until the end of the buffer is reached.
///
/// This value is greater than or equal to the length of the slice returned
/// by `bytes_mut`.
/// by `chunk_mut()`.
///
/// Writing to a `BufMut` may involve allocating more memory on the fly.
/// Implementations may fail before reaching the number of bytes indicated
/// by this method if they encounter an allocation failure.
///
/// # Examples
///
@@ -55,7 +60,7 @@ pub trait BufMut {
/// Advance the internal cursor of the BufMut
///
/// The next call to `bytes_mut` will return a slice starting `cnt` bytes
/// The next call to `chunk_mut` will return a slice starting `cnt` bytes
/// further into the underlying buffer.
///
/// This function is unsafe because there is no guarantee that the bytes
@@ -68,19 +73,14 @@ pub trait BufMut {
///
/// let mut buf = Vec::with_capacity(16);
///
/// unsafe {
/// // MaybeUninit::as_mut_ptr
/// buf.bytes_mut()[0].as_mut_ptr().write(b'h');
/// buf.bytes_mut()[1].as_mut_ptr().write(b'e');
/// // Write some data
/// buf.chunk_mut()[0..2].copy_from_slice(b"he");
/// unsafe { buf.advance_mut(2) };
///
/// buf.advance_mut(2);
/// // write more bytes
/// buf.chunk_mut()[0..3].copy_from_slice(b"llo");
///
/// buf.bytes_mut()[0].as_mut_ptr().write(b'l');
/// buf.bytes_mut()[1].as_mut_ptr().write(b'l');
/// buf.bytes_mut()[2].as_mut_ptr().write(b'o');
///
/// buf.advance_mut(3);
/// }
/// unsafe { buf.advance_mut(3); }
///
/// assert_eq!(5, buf.len());
/// assert_eq!(buf, b"hello");
@@ -139,14 +139,14 @@ pub trait BufMut {
///
/// unsafe {
/// // MaybeUninit::as_mut_ptr
/// buf.bytes_mut()[0].as_mut_ptr().write(b'h');
/// buf.bytes_mut()[1].as_mut_ptr().write(b'e');
/// buf.chunk_mut()[0..].as_mut_ptr().write(b'h');
/// buf.chunk_mut()[1..].as_mut_ptr().write(b'e');
///
/// buf.advance_mut(2);
///
/// buf.bytes_mut()[0].as_mut_ptr().write(b'l');
/// buf.bytes_mut()[1].as_mut_ptr().write(b'l');
/// buf.bytes_mut()[2].as_mut_ptr().write(b'o');
/// buf.chunk_mut()[0..].as_mut_ptr().write(b'l');
/// buf.chunk_mut()[1..].as_mut_ptr().write(b'l');
/// buf.chunk_mut()[2..].as_mut_ptr().write(b'o');
///
/// buf.advance_mut(3);
/// }
@@ -157,54 +157,18 @@ pub trait BufMut {
///
/// # Implementer notes
///
/// This function should never panic. `bytes_mut` should return an empty
/// slice **if and only if** `remaining_mut` returns 0. In other words,
/// `bytes_mut` returning an empty slice implies that `remaining_mut` will
/// return 0 and `remaining_mut` returning 0 implies that `bytes_mut` will
/// This function should never panic. `chunk_mut` should return an empty
/// slice **if and only if** `remaining_mut()` returns 0. In other words,
/// `chunk_mut()` returning an empty slice implies that `remaining_mut()` will
/// return 0 and `remaining_mut()` returning 0 implies that `chunk_mut()` will
/// return an empty slice.
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>];
/// Fills `dst` with potentially multiple mutable slices starting at `self`'s
/// current position.
///
/// If the `BufMut` is backed by disjoint slices of bytes, `bytes_vectored_mut`
/// enables fetching more than one slice at once. `dst` is a slice of
/// mutable `IoSliceMut` references, enabling the slice to be directly used with
/// [`readv`] without any further conversion. The sum of the lengths of all
/// the buffers in `dst` will be less than or equal to
/// `Buf::remaining_mut()`.
///
/// The entries in `dst` will be overwritten, but the data **contained** by
/// the slices **will not** be modified. If `bytes_vectored_mut` does not fill every
/// entry in `dst`, then `dst` is guaranteed to contain all remaining slices
/// in `self.
///
/// This is a lower level function. Most operations are done with other
/// functions.
///
/// # Implementer notes
///
/// This function should never panic. Once the end of the buffer is reached,
/// i.e., `BufMut::remaining_mut` returns 0, calls to `bytes_vectored_mut` must
/// return 0 without mutating `dst`.
///
/// Implementations should also take care to properly handle being called
/// with `dst` being a zero length slice.
///
/// [`readv`]: http://man7.org/linux/man-pages/man2/readv.2.html
#[cfg(feature = "std")]
fn bytes_vectored_mut<'a>(&'a mut self, dst: &mut [IoSliceMut<'a>]) -> usize {
if dst.is_empty() {
return 0;
}
if self.has_remaining_mut() {
dst[0] = IoSliceMut::from(self.bytes_mut());
1
} else {
0
}
}
/// This function may trigger an out-of-memory abort if it tries to allocate
/// memory and fails to do so.
// The `chunk_mut` method was previously called `bytes_mut`. This alias makes the
// rename more easily discoverable.
#[cfg_attr(docsrs, doc(alias = "bytes_mut"))]
fn chunk_mut(&mut self) -> &mut UninitSlice;
/// Transfer bytes into `self` from `src` and advance the cursor by the
/// number of bytes written.
@@ -226,25 +190,27 @@ pub trait BufMut {
/// # Panics
///
/// Panics if `self` does not have enough capacity to contain `src`.
fn put<T: super::Buf>(&mut self, mut src: T) where Self: Sized {
fn put<T: super::Buf>(&mut self, mut src: T)
where
Self: Sized,
{
assert!(self.remaining_mut() >= src.remaining());
while src.has_remaining() {
let l;
unsafe {
let s = src.bytes();
let d = self.bytes_mut();
let s = src.chunk();
let d = self.chunk_mut();
l = cmp::min(s.len(), d.len());
ptr::copy_nonoverlapping(
s.as_ptr(),
d.as_mut_ptr() as *mut u8,
l);
ptr::copy_nonoverlapping(s.as_ptr(), d.as_mut_ptr() as *mut u8, l);
}
src.advance(l);
unsafe { self.advance_mut(l); }
unsafe {
self.advance_mut(l);
}
}
}
@@ -270,25 +236,59 @@ pub trait BufMut {
fn put_slice(&mut self, src: &[u8]) {
let mut off = 0;
assert!(self.remaining_mut() >= src.len(), "buffer overflow; remaining = {}; src = {}", self.remaining_mut(), src.len());
assert!(
self.remaining_mut() >= src.len(),
"buffer overflow; remaining = {}; src = {}",
self.remaining_mut(),
src.len()
);
while off < src.len() {
let cnt;
unsafe {
let dst = self.bytes_mut();
let dst = self.chunk_mut();
cnt = cmp::min(dst.len(), src.len() - off);
ptr::copy_nonoverlapping(
src[off..].as_ptr(),
dst.as_mut_ptr() as *mut u8,
cnt);
ptr::copy_nonoverlapping(src[off..].as_ptr(), dst.as_mut_ptr() as *mut u8, cnt);
off += cnt;
}
unsafe { self.advance_mut(cnt); }
unsafe {
self.advance_mut(cnt);
}
}
}
/// Put `cnt` bytes `val` into `self`.
///
/// Logically equivalent to calling `self.put_u8(val)` `cnt` times, but may work faster.
///
/// `self` must have at least `cnt` remaining capacity.
///
/// ```
/// use bytes::BufMut;
///
/// let mut dst = [0; 6];
///
/// {
/// let mut buf = &mut dst[..];
/// buf.put_bytes(b'a', 4);
///
/// assert_eq!(2, buf.remaining_mut());
/// }
///
/// assert_eq!(b"aaaa\0\0", &dst);
/// ```
///
/// # Panics
///
/// This function panics if there is not enough remaining capacity in
/// `self`.
fn put_bytes(&mut self, val: u8, cnt: usize) {
for _ in 0..cnt {
self.put_u8(val);
}
}
@@ -734,7 +734,7 @@ pub trait BufMut {
self.put_slice(&n.to_le_bytes()[0..nbytes]);
}
/// Writes a signed n-byte integer to `self` in big-endian byte order.
/// Writes low `nbytes` of a signed integer to `self` in big-endian byte order.
///
/// The current position is advanced by `nbytes`.
///
@@ -744,19 +744,19 @@ pub trait BufMut {
/// use bytes::BufMut;
///
/// let mut buf = vec![];
/// buf.put_int(0x010203, 3);
/// buf.put_int(0x0504010203, 3);
/// assert_eq!(buf, b"\x01\x02\x03");
/// ```
///
/// # Panics
///
/// This function panics if there is not enough remaining capacity in
/// `self`.
/// `self` or if `nbytes` is greater than 8.
fn put_int(&mut self, n: i64, nbytes: usize) {
self.put_slice(&n.to_be_bytes()[mem::size_of_val(&n) - nbytes..]);
}
/// Writes a signed n-byte integer to `self` in little-endian byte order.
/// Writes low `nbytes` of a signed integer to `self` in little-endian byte order.
///
/// The current position is advanced by `nbytes`.
///
@@ -766,14 +766,14 @@ pub trait BufMut {
/// use bytes::BufMut;
///
/// let mut buf = vec![];
/// buf.put_int_le(0x010203, 3);
/// buf.put_int_le(0x0504010203, 3);
/// assert_eq!(buf, b"\x03\x02\x01");
/// ```
///
/// # Panics
///
/// This function panics if there is not enough remaining capacity in
/// `self`.
/// `self` or if `nbytes` is greater than 8.
fn put_int_le(&mut self, n: i64, nbytes: usize) {
self.put_slice(&n.to_le_bytes()[0..nbytes]);
}
@@ -869,56 +869,179 @@ pub trait BufMut {
fn put_f64_le(&mut self, n: f64) {
self.put_u64_le(n.to_bits());
}
}
impl<T: BufMut + ?Sized> BufMut for &mut T {
fn remaining_mut(&self) -> usize {
(**self).remaining_mut()
}
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>] {
(**self).bytes_mut()
/// Creates an adaptor which can write at most `limit` bytes to `self`.
///
/// # Examples
///
/// ```
/// use bytes::BufMut;
///
/// let arr = &mut [0u8; 128][..];
/// assert_eq!(arr.remaining_mut(), 128);
///
/// let dst = arr.limit(10);
/// assert_eq!(dst.remaining_mut(), 10);
/// ```
fn limit(self, limit: usize) -> Limit<Self>
where
Self: Sized,
{
limit::new(self, limit)
}
/// Creates an adaptor which implements the `Write` trait for `self`.
///
/// This function returns a new value which implements `Write` by adapting
/// the `Write` trait functions to the `BufMut` trait functions. Given that
/// `BufMut` operations are infallible, none of the `Write` functions will
/// return with `Err`.
///
/// # Examples
///
/// ```
/// use bytes::BufMut;
/// use std::io::Write;
///
/// let mut buf = vec![].writer();
///
/// let num = buf.write(&b"hello world"[..]).unwrap();
/// assert_eq!(11, num);
///
/// let buf = buf.into_inner();
///
/// assert_eq!(*buf, b"hello world"[..]);
/// ```
#[cfg(feature = "std")]
fn bytes_vectored_mut<'b>(&'b mut self, dst: &mut [IoSliceMut<'b>]) -> usize {
(**self).bytes_vectored_mut(dst)
fn writer(self) -> Writer<Self>
where
Self: Sized,
{
writer::new(self)
}
unsafe fn advance_mut(&mut self, cnt: usize) {
(**self).advance_mut(cnt)
/// Creates an adapter which will chain this buffer with another.
///
/// The returned `BufMut` instance will first write to all bytes from
/// `self`. Afterwards, it will write to `next`.
///
/// # Examples
///
/// ```
/// use bytes::BufMut;
///
/// let mut a = [0u8; 5];
/// let mut b = [0u8; 6];
///
/// let mut chain = (&mut a[..]).chain_mut(&mut b[..]);
///
/// chain.put_slice(b"hello world");
///
/// assert_eq!(&a[..], b"hello");
/// assert_eq!(&b[..], b" world");
/// ```
fn chain_mut<U: BufMut>(self, next: U) -> Chain<Self, U>
where
Self: Sized,
{
Chain::new(self, next)
}
}
impl<T: BufMut + ?Sized> BufMut for Box<T> {
fn remaining_mut(&self) -> usize {
(**self).remaining_mut()
}
macro_rules! deref_forward_bufmut {
() => {
fn remaining_mut(&self) -> usize {
(**self).remaining_mut()
}
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>] {
(**self).bytes_mut()
}
fn chunk_mut(&mut self) -> &mut UninitSlice {
(**self).chunk_mut()
}
#[cfg(feature = "std")]
fn bytes_vectored_mut<'b>(&'b mut self, dst: &mut [IoSliceMut<'b>]) -> usize {
(**self).bytes_vectored_mut(dst)
}
unsafe fn advance_mut(&mut self, cnt: usize) {
(**self).advance_mut(cnt)
}
unsafe fn advance_mut(&mut self, cnt: usize) {
(**self).advance_mut(cnt)
}
fn put_slice(&mut self, src: &[u8]) {
(**self).put_slice(src)
}
fn put_u8(&mut self, n: u8) {
(**self).put_u8(n)
}
fn put_i8(&mut self, n: i8) {
(**self).put_i8(n)
}
fn put_u16(&mut self, n: u16) {
(**self).put_u16(n)
}
fn put_u16_le(&mut self, n: u16) {
(**self).put_u16_le(n)
}
fn put_i16(&mut self, n: i16) {
(**self).put_i16(n)
}
fn put_i16_le(&mut self, n: i16) {
(**self).put_i16_le(n)
}
fn put_u32(&mut self, n: u32) {
(**self).put_u32(n)
}
fn put_u32_le(&mut self, n: u32) {
(**self).put_u32_le(n)
}
fn put_i32(&mut self, n: i32) {
(**self).put_i32(n)
}
fn put_i32_le(&mut self, n: i32) {
(**self).put_i32_le(n)
}
fn put_u64(&mut self, n: u64) {
(**self).put_u64(n)
}
fn put_u64_le(&mut self, n: u64) {
(**self).put_u64_le(n)
}
fn put_i64(&mut self, n: i64) {
(**self).put_i64(n)
}
fn put_i64_le(&mut self, n: i64) {
(**self).put_i64_le(n)
}
};
}
impl BufMut for &mut [u8] {
unsafe impl<T: BufMut + ?Sized> BufMut for &mut T {
deref_forward_bufmut!();
}
unsafe impl<T: BufMut + ?Sized> BufMut for Box<T> {
deref_forward_bufmut!();
}
unsafe impl BufMut for &mut [u8] {
#[inline]
fn remaining_mut(&self) -> usize {
self.len()
}
#[inline]
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>] {
// MaybeUninit is repr(transparent), so safe to transmute
unsafe { mem::transmute(&mut **self) }
fn chunk_mut(&mut self) -> &mut UninitSlice {
// UninitSlice is repr(transparent), so safe to transmute
unsafe { &mut *(*self as *mut [u8] as *mut _) }
}
#[inline]
@@ -927,31 +1050,48 @@ impl BufMut for &mut [u8] {
let (_, b) = core::mem::replace(self, &mut []).split_at_mut(cnt);
*self = b;
}
#[inline]
fn put_slice(&mut self, src: &[u8]) {
self[..src.len()].copy_from_slice(src);
unsafe {
self.advance_mut(src.len());
}
}
fn put_bytes(&mut self, val: u8, cnt: usize) {
assert!(self.remaining_mut() >= cnt);
unsafe {
ptr::write_bytes(self.as_mut_ptr(), val, cnt);
self.advance_mut(cnt);
}
}
}
impl BufMut for Vec<u8> {
unsafe impl BufMut for Vec<u8> {
#[inline]
fn remaining_mut(&self) -> usize {
usize::MAX - self.len()
// A vector can never have more than isize::MAX bytes
core::isize::MAX as usize - self.len()
}
#[inline]
unsafe fn advance_mut(&mut self, cnt: usize) {
let len = self.len();
let remaining = self.capacity() - len;
if cnt > remaining {
// Reserve additional capacity, and ensure that the total length
// will not overflow usize.
self.reserve(cnt);
}
assert!(
cnt <= remaining,
"cannot advance past `remaining_mut`: {:?} <= {:?}",
cnt,
remaining
);
self.set_len(len + cnt);
}
#[inline]
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>] {
use core::slice;
fn chunk_mut(&mut self) -> &mut UninitSlice {
if self.capacity() == self.len() {
self.reserve(64); // Grow the vec
}
@@ -959,54 +1099,44 @@ impl BufMut for Vec<u8> {
let cap = self.capacity();
let len = self.len();
let ptr = self.as_mut_ptr() as *mut MaybeUninit<u8>;
unsafe {
&mut slice::from_raw_parts_mut(ptr, cap)[len..]
let ptr = self.as_mut_ptr();
unsafe { &mut UninitSlice::from_raw_parts_mut(ptr, cap)[len..] }
}
// Specialize these methods so they can skip checking `remaining_mut`
// and `advance_mut`.
fn put<T: super::Buf>(&mut self, mut src: T)
where
Self: Sized,
{
// In case the src isn't contiguous, reserve upfront
self.reserve(src.remaining());
while src.has_remaining() {
let l;
// a block to contain the src.bytes() borrow
{
let s = src.chunk();
l = s.len();
self.extend_from_slice(s);
}
src.advance(l);
}
}
#[inline]
fn put_slice(&mut self, src: &[u8]) {
self.extend_from_slice(src);
}
fn put_bytes(&mut self, val: u8, cnt: usize) {
let new_len = self.len().checked_add(cnt).unwrap();
self.resize(new_len, val);
}
}
// The existence of this function makes the compiler catch if the BufMut
// trait is "object-safe" or not.
fn _assert_trait_object(_b: &dyn BufMut) {}
// ===== impl IoSliceMut =====
/// A buffer type used for `readv`.
///
/// This is a wrapper around an `std::io::IoSliceMut`, but does not expose
/// the inner bytes in a safe API, as they may point at uninitialized memory.
///
/// This is `repr(transparent)` of the `std::io::IoSliceMut`, so it is valid to
/// transmute them. However, as the memory might be uninitialized, care must be
/// taken to not *read* the internal bytes, only *write* to them.
#[repr(transparent)]
#[cfg(feature = "std")]
pub struct IoSliceMut<'a>(std::io::IoSliceMut<'a>);
#[cfg(feature = "std")]
impl fmt::Debug for IoSliceMut<'_> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("IoSliceMut")
.field("len", &self.0.len())
.finish()
}
}
#[cfg(feature = "std")]
impl<'a> From<&'a mut [u8]> for IoSliceMut<'a> {
fn from(buf: &'a mut [u8]) -> IoSliceMut<'a> {
IoSliceMut(std::io::IoSliceMut::new(buf))
}
}
#[cfg(feature = "std")]
impl<'a> From<&'a mut [MaybeUninit<u8>]> for IoSliceMut<'a> {
fn from(buf: &'a mut [MaybeUninit<u8>]) -> IoSliceMut<'a> {
IoSliceMut(std::io::IoSliceMut::new(unsafe {
// We don't look at the contents, and `std::io::IoSliceMut`
// doesn't either.
mem::transmute::<&'a mut [MaybeUninit<u8>], &'a mut [u8]>(buf)
}))
}
}
+53 -44
View File
@@ -1,12 +1,8 @@
use crate::{Buf, BufMut};
use crate::buf::IntoIter;
use core::mem::MaybeUninit;
use crate::buf::{IntoIter, UninitSlice};
use crate::{Buf, BufMut, Bytes};
#[cfg(feature = "std")]
use std::io::{IoSlice};
#[cfg(feature = "std")]
use crate::buf::IoSliceMut;
use std::io::IoSlice;
/// A `Chain` sequences two buffers.
///
@@ -20,12 +16,12 @@ use crate::buf::IoSliceMut;
/// # Examples
///
/// ```
/// use bytes::{Bytes, Buf, buf::BufExt};
/// use bytes::{Bytes, Buf};
///
/// let mut buf = (&b"hello "[..])
/// .chain(&b"world"[..]);
///
/// let full: Bytes = buf.to_bytes();
/// let full: Bytes = buf.copy_to_bytes(11);
/// assert_eq!(full[..], b"hello world"[..]);
/// ```
///
@@ -40,11 +36,8 @@ pub struct Chain<T, U> {
impl<T, U> Chain<T, U> {
/// Creates a new `Chain` sequencing the provided values.
pub fn new(a: T, b: U) -> Chain<T, U> {
Chain {
a,
b,
}
pub(crate) fn new(a: T, b: U) -> Chain<T, U> {
Chain { a, b }
}
/// Gets a reference to the first underlying `Buf`.
@@ -52,7 +45,7 @@ impl<T, U> Chain<T, U> {
/// # Examples
///
/// ```
/// use bytes::buf::BufExt;
/// use bytes::Buf;
///
/// let buf = (&b"hello"[..])
/// .chain(&b"world"[..]);
@@ -68,14 +61,14 @@ impl<T, U> Chain<T, U> {
/// # Examples
///
/// ```
/// use bytes::{Buf, buf::BufExt};
/// use bytes::Buf;
///
/// let mut buf = (&b"hello"[..])
/// .chain(&b"world"[..]);
///
/// buf.first_mut().advance(1);
///
/// let full = buf.to_bytes();
/// let full = buf.copy_to_bytes(9);
/// assert_eq!(full, b"elloworld"[..]);
/// ```
pub fn first_mut(&mut self) -> &mut T {
@@ -87,7 +80,7 @@ impl<T, U> Chain<T, U> {
/// # Examples
///
/// ```
/// use bytes::buf::BufExt;
/// use bytes::Buf;
///
/// let buf = (&b"hello"[..])
/// .chain(&b"world"[..]);
@@ -103,14 +96,14 @@ impl<T, U> Chain<T, U> {
/// # Examples
///
/// ```
/// use bytes::{Buf, buf::BufExt};
/// use bytes::Buf;
///
/// let mut buf = (&b"hello "[..])
/// .chain(&b"world"[..]);
///
/// buf.last_mut().advance(1);
///
/// let full = buf.to_bytes();
/// let full = buf.copy_to_bytes(10);
/// assert_eq!(full, b"hello orld"[..]);
/// ```
pub fn last_mut(&mut self) -> &mut U {
@@ -122,7 +115,7 @@ impl<T, U> Chain<T, U> {
/// # Examples
///
/// ```
/// use bytes::buf::BufExt;
/// use bytes::Buf;
///
/// let chain = (&b"hello"[..])
/// .chain(&b"world"[..]);
@@ -137,18 +130,19 @@ impl<T, U> Chain<T, U> {
}
impl<T, U> Buf for Chain<T, U>
where T: Buf,
U: Buf,
where
T: Buf,
U: Buf,
{
fn remaining(&self) -> usize {
self.a.remaining() + self.b.remaining()
self.a.remaining().checked_add(self.b.remaining()).unwrap()
}
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
if self.a.has_remaining() {
self.a.bytes()
self.a.chunk()
} else {
self.b.bytes()
self.b.chunk()
}
}
@@ -171,26 +165,48 @@ impl<T, U> Buf for Chain<T, U>
}
#[cfg(feature = "std")]
fn bytes_vectored<'a>(&'a self, dst: &mut [IoSlice<'a>]) -> usize {
let mut n = self.a.bytes_vectored(dst);
n += self.b.bytes_vectored(&mut dst[n..]);
fn chunks_vectored<'a>(&'a self, dst: &mut [IoSlice<'a>]) -> usize {
let mut n = self.a.chunks_vectored(dst);
n += self.b.chunks_vectored(&mut dst[n..]);
n
}
fn copy_to_bytes(&mut self, len: usize) -> Bytes {
let a_rem = self.a.remaining();
if a_rem >= len {
self.a.copy_to_bytes(len)
} else if a_rem == 0 {
self.b.copy_to_bytes(len)
} else {
assert!(
len - a_rem <= self.b.remaining(),
"`len` greater than remaining"
);
let mut ret = crate::BytesMut::with_capacity(len);
ret.put(&mut self.a);
ret.put((&mut self.b).take(len - a_rem));
ret.freeze()
}
}
}
impl<T, U> BufMut for Chain<T, U>
where T: BufMut,
U: BufMut,
unsafe impl<T, U> BufMut for Chain<T, U>
where
T: BufMut,
U: BufMut,
{
fn remaining_mut(&self) -> usize {
self.a.remaining_mut() + self.b.remaining_mut()
self.a
.remaining_mut()
.checked_add(self.b.remaining_mut())
.unwrap()
}
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>] {
fn chunk_mut(&mut self) -> &mut UninitSlice {
if self.a.has_remaining_mut() {
self.a.bytes_mut()
self.a.chunk_mut()
} else {
self.b.bytes_mut()
self.b.chunk_mut()
}
}
@@ -211,13 +227,6 @@ impl<T, U> BufMut for Chain<T, U>
self.b.advance_mut(cnt);
}
#[cfg(feature = "std")]
fn bytes_vectored_mut<'a>(&'a mut self, dst: &mut [IoSliceMut<'a>]) -> usize {
let mut n = self.a.bytes_vectored_mut(dst);
n += self.b.bytes_vectored_mut(&mut dst[n..]);
n
}
}
impl<T, U> IntoIterator for Chain<T, U>
-176
View File
@@ -1,176 +0,0 @@
//! Extra utilities for `Buf` and `BufMut` types.
use super::{Buf, BufMut};
mod chain;
mod limit;
#[cfg(feature = "std")]
mod reader;
mod take;
#[cfg(feature = "std")]
mod writer;
pub use self::limit::Limit;
pub use self::take::Take;
pub use self::chain::Chain;
#[cfg(feature = "std")]
pub use self::{reader::Reader, writer::Writer};
/// Extra methods for implementations of `Buf`.
pub trait BufExt: Buf {
/// Creates an adaptor which will read at most `limit` bytes from `self`.
///
/// This function returns a new instance of `Buf` which will read at most
/// `limit` bytes.
///
/// # Examples
///
/// ```
/// use bytes::{Buf, BufMut, buf::BufExt};
///
/// let mut buf = b"hello world"[..].take(5);
/// let mut dst = vec![];
///
/// dst.put(&mut buf);
/// assert_eq!(dst, b"hello");
///
/// let mut buf = buf.into_inner();
/// dst.clear();
/// dst.put(&mut buf);
/// assert_eq!(dst, b" world");
/// ```
fn take(self, limit: usize) -> Take<Self>
where Self: Sized
{
take::new(self, limit)
}
/// Creates an adaptor which will chain this buffer with another.
///
/// The returned `Buf` instance will first consume all bytes from `self`.
/// Afterwards the output is equivalent to the output of next.
///
/// # Examples
///
/// ```
/// use bytes::{Buf, buf::BufExt};
///
/// let mut chain = b"hello "[..].chain(&b"world"[..]);
///
/// let full = chain.to_bytes();
/// assert_eq!(full.bytes(), b"hello world");
/// ```
fn chain<U: Buf>(self, next: U) -> Chain<Self, U>
where Self: Sized
{
Chain::new(self, next)
}
/// Creates an adaptor which implements the `Read` trait for `self`.
///
/// This function returns a new value which implements `Read` by adapting
/// the `Read` trait functions to the `Buf` trait functions. Given that
/// `Buf` operations are infallible, none of the `Read` functions will
/// return with `Err`.
///
/// # Examples
///
/// ```
/// use bytes::{Buf, Bytes, buf::BufExt};
/// use std::io::Read;
///
/// let buf = Bytes::from("hello world");
///
/// let mut reader = buf.reader();
/// let mut dst = [0; 1024];
///
/// let num = reader.read(&mut dst).unwrap();
///
/// assert_eq!(11, num);
/// assert_eq!(&dst[..11], &b"hello world"[..]);
/// ```
#[cfg(feature = "std")]
fn reader(self) -> Reader<Self> where Self: Sized {
reader::new(self)
}
}
impl<B: Buf + ?Sized> BufExt for B {}
/// Extra methods for implementations of `BufMut`.
pub trait BufMutExt: BufMut {
/// Creates an adaptor which can write at most `limit` bytes to `self`.
///
/// # Examples
///
/// ```
/// use bytes::{BufMut, buf::BufMutExt};
///
/// let arr = &mut [0u8; 128][..];
/// assert_eq!(arr.remaining_mut(), 128);
///
/// let dst = arr.limit(10);
/// assert_eq!(dst.remaining_mut(), 10);
/// ```
fn limit(self, limit: usize) -> Limit<Self>
where Self: Sized
{
limit::new(self, limit)
}
/// Creates an adaptor which implements the `Write` trait for `self`.
///
/// This function returns a new value which implements `Write` by adapting
/// the `Write` trait functions to the `BufMut` trait functions. Given that
/// `BufMut` operations are infallible, none of the `Write` functions will
/// return with `Err`.
///
/// # Examples
///
/// ```
/// use bytes::{BufMut, buf::BufMutExt};
/// use std::io::Write;
///
/// let mut buf = vec![].writer();
///
/// let num = buf.write(&b"hello world"[..]).unwrap();
/// assert_eq!(11, num);
///
/// let buf = buf.into_inner();
///
/// assert_eq!(*buf, b"hello world"[..]);
/// ```
#[cfg(feature = "std")]
fn writer(self) -> Writer<Self> where Self: Sized {
writer::new(self)
}
/// Creates an adapter which will chain this buffer with another.
///
/// The returned `BufMut` instance will first write to all bytes from
/// `self`. Afterwards, it will write to `next`.
///
/// # Examples
///
/// ```
/// use bytes::{BufMut, buf::BufMutExt};
///
/// let mut a = [0u8; 5];
/// let mut b = [0u8; 6];
///
/// let mut chain = (&mut a[..]).chain_mut(&mut b[..]);
///
/// chain.put_slice(b"hello world");
///
/// assert_eq!(&a[..], b"hello");
/// assert_eq!(&b[..], b" world");
/// ```
fn chain_mut<U: BufMut>(self, next: U) -> Chain<Self, U>
where Self: Sized
{
Chain::new(self, next)
}
}
impl<B: BufMut + ?Sized> BufMutExt for B {}
+7 -8
View File
@@ -9,7 +9,7 @@ use crate::Buf;
/// Basic usage:
///
/// ```
/// use bytes::{Buf, Bytes};
/// use bytes::Bytes;
///
/// let buf = Bytes::from(&b"abc"[..]);
/// let mut iter = buf.into_iter();
@@ -33,20 +33,20 @@ impl<T> IntoIter<T> {
/// # Examples
///
/// ```
/// use bytes::{Buf, Bytes};
/// use bytes::buf::IntoIter;
/// use bytes::Bytes;
///
/// let buf = Bytes::from_static(b"abc");
/// let mut iter = IntoIter::new(buf);
/// let mut iter = buf.into_iter();
///
/// assert_eq!(iter.next(), Some(b'a'));
/// assert_eq!(iter.next(), Some(b'b'));
/// assert_eq!(iter.next(), Some(b'c'));
/// assert_eq!(iter.next(), None);
/// ```
pub fn new(inner: T) -> IntoIter<T> {
pub(crate) fn new(inner: T) -> IntoIter<T> {
IntoIter { inner }
}
/// Consumes this `IntoIter`, returning the underlying value.
///
/// # Examples
@@ -109,7 +109,6 @@ impl<T> IntoIter<T> {
}
}
impl<T: Buf> Iterator for IntoIter<T> {
type Item = u8;
@@ -118,7 +117,7 @@ impl<T: Buf> Iterator for IntoIter<T> {
return None;
}
let b = self.inner.bytes()[0];
let b = self.inner.chunk()[0];
self.inner.advance(1);
Some(b)
@@ -130,4 +129,4 @@ impl<T: Buf> Iterator for IntoIter<T> {
}
}
impl<T: Buf> ExactSizeIterator for IntoIter<T> { }
impl<T: Buf> ExactSizeIterator for IntoIter<T> {}
+6 -8
View File
@@ -1,6 +1,7 @@
use crate::buf::UninitSlice;
use crate::BufMut;
use core::{cmp, mem::MaybeUninit};
use core::cmp;
/// A `BufMut` adapter which limits the amount of bytes that can be written
/// to an underlying buffer.
@@ -11,10 +12,7 @@ pub struct Limit<T> {
}
pub(super) fn new<T>(inner: T, limit: usize) -> Limit<T> {
Limit {
inner,
limit,
}
Limit { inner, limit }
}
impl<T> Limit<T> {
@@ -58,13 +56,13 @@ impl<T> Limit<T> {
}
}
impl<T: BufMut> BufMut for Limit<T> {
unsafe impl<T: BufMut> BufMut for Limit<T> {
fn remaining_mut(&self) -> usize {
cmp::min(self.inner.remaining_mut(), self.limit)
}
fn bytes_mut(&mut self) -> &mut [MaybeUninit<u8>] {
let bytes = self.inner.bytes_mut();
fn chunk_mut(&mut self) -> &mut UninitSlice {
let bytes = self.inner.chunk_mut();
let end = cmp::min(bytes.len(), self.limit);
&mut bytes[..end]
}
+14 -4
View File
@@ -18,14 +18,24 @@
mod buf_impl;
mod buf_mut;
pub mod ext;
mod chain;
mod iter;
mod limit;
#[cfg(feature = "std")]
mod reader;
mod take;
mod uninit_slice;
mod vec_deque;
#[cfg(feature = "std")]
mod writer;
pub use self::buf_impl::Buf;
pub use self::buf_mut::BufMut;
pub use self::ext::{BufExt, BufMutExt};
#[cfg(feature = "std")]
pub use self::buf_mut::IoSliceMut;
pub use self::chain::Chain;
pub use self::iter::IntoIter;
pub use self::limit::Limit;
pub use self::take::Take;
pub use self::uninit_slice::UninitSlice;
#[cfg(feature = "std")]
pub use self::{reader::Reader, writer::Writer};
+5 -5
View File
@@ -1,4 +1,4 @@
use crate::{Buf};
use crate::Buf;
use std::{cmp, io};
@@ -24,9 +24,9 @@ impl<B: Buf> Reader<B> {
/// # Examples
///
/// ```rust
/// use bytes::buf::BufExt;
/// use bytes::Buf;
///
/// let mut buf = b"hello world".reader();
/// let buf = b"hello world".reader();
///
/// assert_eq!(b"hello world", buf.get_ref());
/// ```
@@ -46,7 +46,7 @@ impl<B: Buf> Reader<B> {
/// # Examples
///
/// ```rust
/// use bytes::{Buf, buf::BufExt};
/// use bytes::Buf;
/// use std::io;
///
/// let mut buf = b"hello world".reader();
@@ -73,7 +73,7 @@ impl<B: Buf + Sized> io::Read for Reader<B> {
impl<B: Buf + Sized> io::BufRead for Reader<B> {
fn fill_buf(&mut self) -> io::Result<&[u8]> {
Ok(self.buf.bytes())
Ok(self.buf.chunk())
}
fn consume(&mut self, amt: usize) {
self.buf.advance(amt)
+18 -13
View File
@@ -1,4 +1,4 @@
use crate::Buf;
use crate::{Buf, Bytes};
use core::cmp;
@@ -13,10 +13,7 @@ pub struct Take<T> {
}
pub fn new<T>(inner: T, limit: usize) -> Take<T> {
Take {
inner,
limit,
}
Take { inner, limit }
}
impl<T> Take<T> {
@@ -25,7 +22,7 @@ impl<T> Take<T> {
/// # Examples
///
/// ```rust
/// use bytes::buf::{Buf, BufMut, BufExt};
/// use bytes::{Buf, BufMut};
///
/// let mut buf = b"hello world".take(2);
/// let mut dst = vec![];
@@ -50,9 +47,9 @@ impl<T> Take<T> {
/// # Examples
///
/// ```rust
/// use bytes::{Buf, buf::BufExt};
/// use bytes::Buf;
///
/// let mut buf = b"hello world".take(2);
/// let buf = b"hello world".take(2);
///
/// assert_eq!(11, buf.get_ref().remaining());
/// ```
@@ -67,7 +64,7 @@ impl<T> Take<T> {
/// # Examples
///
/// ```rust
/// use bytes::{Buf, BufMut, buf::BufExt};
/// use bytes::{Buf, BufMut};
///
/// let mut buf = b"hello world".take(2);
/// let mut dst = vec![];
@@ -91,7 +88,7 @@ impl<T> Take<T> {
/// # Examples
///
/// ```rust
/// use bytes::{Buf, buf::BufExt};
/// use bytes::Buf;
///
/// let mut buf = b"hello world".take(2);
///
@@ -113,7 +110,7 @@ impl<T> Take<T> {
/// # Examples
///
/// ```rust
/// use bytes::{Buf, BufMut, buf::BufExt};
/// use bytes::{Buf, BufMut};
///
/// let mut buf = b"hello world".take(2);
/// let mut dst = vec![];
@@ -137,8 +134,8 @@ impl<T: Buf> Buf for Take<T> {
cmp::min(self.inner.remaining(), self.limit)
}
fn bytes(&self) -> &[u8] {
let bytes = self.inner.bytes();
fn chunk(&self) -> &[u8] {
let bytes = self.inner.chunk();
&bytes[..cmp::min(bytes.len(), self.limit)]
}
@@ -147,4 +144,12 @@ impl<T: Buf> Buf for Take<T> {
self.inner.advance(cnt);
self.limit -= cnt;
}
fn copy_to_bytes(&mut self, len: usize) -> Bytes {
assert!(len <= self.remaining(), "`len` greater than remaining");
let r = self.inner.copy_to_bytes(len);
self.limit -= len;
r
}
}
+183
View File
@@ -0,0 +1,183 @@
use core::fmt;
use core::mem::MaybeUninit;
use core::ops::{
Index, IndexMut, Range, RangeFrom, RangeFull, RangeInclusive, RangeTo, RangeToInclusive,
};
/// Uninitialized byte slice.
///
/// Returned by `BufMut::chunk_mut()`, the referenced byte slice may be
/// uninitialized. The wrapper provides safe access without introducing
/// undefined behavior.
///
/// The safety invariants of this wrapper are:
///
/// 1. Reading from an `UninitSlice` is undefined behavior.
/// 2. Writing uninitialized bytes to an `UninitSlice` is undefined behavior.
///
/// The difference between `&mut UninitSlice` and `&mut [MaybeUninit<u8>]` is
/// that it is possible in safe code to write uninitialized bytes to an
/// `&mut [MaybeUninit<u8>]`, which this type prohibits.
#[repr(transparent)]
pub struct UninitSlice([MaybeUninit<u8>]);
impl UninitSlice {
/// Create a `&mut UninitSlice` from a pointer and a length.
///
/// # Safety
///
/// The caller must ensure that `ptr` references a valid memory region owned
/// by the caller representing a byte slice for the duration of `'a`.
///
/// # Examples
///
/// ```
/// use bytes::buf::UninitSlice;
///
/// let bytes = b"hello world".to_vec();
/// let ptr = bytes.as_ptr() as *mut _;
/// let len = bytes.len();
///
/// let slice = unsafe { UninitSlice::from_raw_parts_mut(ptr, len) };
/// ```
#[inline]
pub unsafe fn from_raw_parts_mut<'a>(ptr: *mut u8, len: usize) -> &'a mut UninitSlice {
let maybe_init: &mut [MaybeUninit<u8>] =
core::slice::from_raw_parts_mut(ptr as *mut _, len);
&mut *(maybe_init as *mut [MaybeUninit<u8>] as *mut UninitSlice)
}
/// Write a single byte at the specified offset.
///
/// # Panics
///
/// The function panics if `index` is out of bounds.
///
/// # Examples
///
/// ```
/// use bytes::buf::UninitSlice;
///
/// let mut data = [b'f', b'o', b'o'];
/// let slice = unsafe { UninitSlice::from_raw_parts_mut(data.as_mut_ptr(), 3) };
///
/// slice.write_byte(0, b'b');
///
/// assert_eq!(b"boo", &data[..]);
/// ```
#[inline]
pub fn write_byte(&mut self, index: usize, byte: u8) {
assert!(index < self.len());
unsafe { self[index..].as_mut_ptr().write(byte) }
}
/// Copies bytes from `src` into `self`.
///
/// The length of `src` must be the same as `self`.
///
/// # Panics
///
/// The function panics if `src` has a different length than `self`.
///
/// # Examples
///
/// ```
/// use bytes::buf::UninitSlice;
///
/// let mut data = [b'f', b'o', b'o'];
/// let slice = unsafe { UninitSlice::from_raw_parts_mut(data.as_mut_ptr(), 3) };
///
/// slice.copy_from_slice(b"bar");
///
/// assert_eq!(b"bar", &data[..]);
/// ```
#[inline]
pub fn copy_from_slice(&mut self, src: &[u8]) {
use core::ptr;
assert_eq!(self.len(), src.len());
unsafe {
ptr::copy_nonoverlapping(src.as_ptr(), self.as_mut_ptr(), self.len());
}
}
/// Return a raw pointer to the slice's buffer.
///
/// # Safety
///
/// The caller **must not** read from the referenced memory and **must not**
/// write **uninitialized** bytes to the slice either.
///
/// # Examples
///
/// ```
/// use bytes::BufMut;
///
/// let mut data = [0, 1, 2];
/// let mut slice = &mut data[..];
/// let ptr = BufMut::chunk_mut(&mut slice).as_mut_ptr();
/// ```
#[inline]
pub fn as_mut_ptr(&mut self) -> *mut u8 {
self.0.as_mut_ptr() as *mut _
}
/// Returns the number of bytes in the slice.
///
/// # Examples
///
/// ```
/// use bytes::BufMut;
///
/// let mut data = [0, 1, 2];
/// let mut slice = &mut data[..];
/// let len = BufMut::chunk_mut(&mut slice).len();
///
/// assert_eq!(len, 3);
/// ```
#[inline]
pub fn len(&self) -> usize {
self.0.len()
}
}
impl fmt::Debug for UninitSlice {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt.debug_struct("UninitSlice[...]").finish()
}
}
macro_rules! impl_index {
($($t:ty),*) => {
$(
impl Index<$t> for UninitSlice {
type Output = UninitSlice;
#[inline]
fn index(&self, index: $t) -> &UninitSlice {
let maybe_uninit: &[MaybeUninit<u8>] = &self.0[index];
unsafe { &*(maybe_uninit as *const [MaybeUninit<u8>] as *const UninitSlice) }
}
}
impl IndexMut<$t> for UninitSlice {
#[inline]
fn index_mut(&mut self, index: $t) -> &mut UninitSlice {
let maybe_uninit: &mut [MaybeUninit<u8>] = &mut self.0[index];
unsafe { &mut *(maybe_uninit as *mut [MaybeUninit<u8>] as *mut UninitSlice) }
}
}
)*
};
}
impl_index!(
Range<usize>,
RangeFrom<usize>,
RangeFull,
RangeInclusive<usize>,
RangeTo<usize>,
RangeToInclusive<usize>
);
+1 -1
View File
@@ -7,7 +7,7 @@ impl Buf for VecDeque<u8> {
self.len()
}
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
let (s1, s2) = self.as_slices();
if s1.is_empty() {
s2
+4 -4
View File
@@ -24,9 +24,9 @@ impl<B: BufMut> Writer<B> {
/// # Examples
///
/// ```rust
/// use bytes::buf::BufMutExt;
/// use bytes::BufMut;
///
/// let mut buf = Vec::with_capacity(1024).writer();
/// let buf = Vec::with_capacity(1024).writer();
///
/// assert_eq!(1024, buf.get_ref().capacity());
/// ```
@@ -41,7 +41,7 @@ impl<B: BufMut> Writer<B> {
/// # Examples
///
/// ```rust
/// use bytes::buf::BufMutExt;
/// use bytes::BufMut;
///
/// let mut buf = vec![].writer();
///
@@ -58,7 +58,7 @@ impl<B: BufMut> Writer<B> {
/// # Examples
///
/// ```rust
/// use bytes::buf::BufMutExt;
/// use bytes::BufMut;
/// use std::io;
///
/// let mut buf = vec![].writer();
+320 -125
View File
@@ -1,24 +1,32 @@
use core::{cmp, fmt, hash, mem, ptr, slice, usize};
use core::iter::{FromIterator};
use core::iter::FromIterator;
use core::ops::{Deref, RangeBounds};
use core::{cmp, fmt, hash, mem, ptr, slice, usize};
use alloc::{vec::Vec, string::String, boxed::Box, borrow::Borrow};
use alloc::{borrow::Borrow, boxed::Box, string::String, vec::Vec};
use crate::Buf;
use crate::buf::IntoIter;
use crate::debug;
#[allow(unused)]
use crate::loom::sync::atomic::AtomicMut;
use crate::loom::sync::atomic::{self, AtomicPtr, AtomicUsize, Ordering};
use crate::Buf;
/// A reference counted contiguous slice of memory.
/// A cheaply cloneable and sliceable chunk of contiguous memory.
///
/// `Bytes` is an efficient container for storing and operating on contiguous
/// slices of memory. It is intended for use primarily in networking code, but
/// could have applications elsewhere as well.
///
/// `Bytes` values facilitate zero-copy network programming by allowing multiple
/// `Bytes` objects to point to the same underlying memory. This is managed by
/// using a reference count to track when the memory is no longer needed and can
/// be freed.
/// `Bytes` objects to point to the same underlying memory.
///
/// `Bytes` does not have a single implementation. It is an interface, whose
/// exact behavior is implemented through dynamic dispatch in several underlying
/// implementations of `Bytes`.
///
/// All `Bytes` implementations must fulfill the following requirements:
/// - They are cheaply cloneable and thereby shareable between an unlimited amount
/// of components, for example by modifying a reference count.
/// - Instances can be sliced to refer to a subset of the the original buffer.
///
/// ```
/// use bytes::Bytes;
@@ -40,17 +48,33 @@ use crate::loom::sync::atomic::{self, AtomicPtr, AtomicUsize, Ordering};
/// to track information about which segment of the underlying memory the
/// `Bytes` handle has access to.
///
/// `Bytes` keeps both a pointer to the shared `Arc` containing the full memory
/// `Bytes` keeps both a pointer to the shared state containing the full memory
/// slice and a pointer to the start of the region visible by the handle.
/// `Bytes` also tracks the length of its view into the memory.
///
/// # Sharing
///
/// The memory itself is reference counted, and multiple `Bytes` objects may
/// point to the same region. Each `Bytes` handle point to different sections within
/// the memory region, and `Bytes` handle may or may not have overlapping views
/// `Bytes` contains a vtable, which allows implementations of `Bytes` to define
/// how sharing/cloneing is implemented in detail.
/// When `Bytes::clone()` is called, `Bytes` will call the vtable function for
/// cloning the backing storage in order to share it behind between multiple
/// `Bytes` instances.
///
/// For `Bytes` implementations which refer to constant memory (e.g. created
/// via `Bytes::from_static()`) the cloning implementation will be a no-op.
///
/// For `Bytes` implementations which point to a reference counted shared storage
/// (e.g. an `Arc<[u8]>`), sharing will be implemented by increasing the
/// the reference count.
///
/// Due to this mechanism, multiple `Bytes` instances may point to the same
/// shared memory region.
/// Each `Bytes` instance can point to different sections within that
/// memory region, and `Bytes` instances may or may not have overlapping views
/// into the memory.
///
/// The following diagram visualizes a scenario where 2 `Bytes` instances make
/// use of an `Arc`-based backing storage, and provide access to different views:
///
/// ```text
///
@@ -96,8 +120,18 @@ impl Bytes {
/// assert_eq!(&b[..], b"");
/// ```
#[inline]
#[cfg(not(all(loom, test)))]
pub const fn new() -> Bytes {
// Make it a named const to work around
// "unsizing casts are not allowed in const fn"
const EMPTY: &[u8] = &[];
Bytes::from_static(EMPTY)
}
#[cfg(all(loom, test))]
pub fn new() -> Bytes {
Bytes::from_static(b"")
const EMPTY: &[u8] = &[];
Bytes::from_static(EMPTY)
}
/// Creates a new `Bytes` from a static slice.
@@ -164,8 +198,7 @@ impl Bytes {
self.len == 0
}
///Creates `Bytes` instance from slice, by copying it.
/// Creates `Bytes` instance from slice, by copying it.
pub fn copy_from_slice(data: &[u8]) -> Self {
data.to_vec().into()
}
@@ -204,19 +237,28 @@ impl Bytes {
};
let end = match range.end_bound() {
Bound::Included(&n) => n + 1,
Bound::Included(&n) => n.checked_add(1).expect("out of range"),
Bound::Excluded(&n) => n,
Bound::Unbounded => len,
};
assert!(begin <= end);
assert!(end <= len);
assert!(
begin <= end,
"range start must not be greater than end: {:?} <= {:?}",
begin,
end,
);
assert!(
end <= len,
"range end out of bounds: {:?} <= {:?}",
end,
len,
);
if end == begin {
return Bytes::new();
}
let mut ret = self.clone();
ret.len = end - begin;
@@ -251,14 +293,32 @@ impl Bytes {
/// Requires that the given `sub` slice is in fact contained within the
/// `Bytes` buffer; otherwise this function will panic.
pub fn slice_ref(&self, subset: &[u8]) -> Bytes {
// Empty slice and empty Bytes may have their pointers reset
// so explicitly allow empty slice to be a subslice of any slice.
if subset.is_empty() {
return Bytes::new();
}
let bytes_p = self.as_ptr() as usize;
let bytes_len = self.len();
let sub_p = subset.as_ptr() as usize;
let sub_len = subset.len();
assert!(sub_p >= bytes_p);
assert!(sub_p + sub_len <= bytes_p + bytes_len);
assert!(
sub_p >= bytes_p,
"subset pointer ({:p}) is smaller than self pointer ({:p})",
sub_p as *const u8,
bytes_p as *const u8,
);
assert!(
sub_p + sub_len <= bytes_p + bytes_len,
"subset is out of bounds: self = ({:p}, {}), subset = ({:p}, {})",
bytes_p as *const u8,
bytes_len,
sub_p as *const u8,
sub_len,
);
let sub_offset = sub_p - bytes_p;
@@ -288,8 +348,14 @@ impl Bytes {
/// # Panics
///
/// Panics if `at > len`.
#[must_use = "consider Bytes::truncate if you don't need the other half"]
pub fn split_off(&mut self, at: usize) -> Bytes {
assert!(at <= self.len());
assert!(
at <= self.len(),
"split_off out of bounds: {:?} <= {:?}",
at,
self.len(),
);
if at == self.len() {
return Bytes::new();
@@ -331,8 +397,14 @@ impl Bytes {
/// # Panics
///
/// Panics if `at > len`.
#[must_use = "consider Bytes::advance if you don't need the other half"]
pub fn split_to(&mut self, at: usize) -> Bytes {
assert!(at <= self.len());
assert!(
at <= self.len(),
"split_to out of bounds: {:?} <= {:?}",
at,
self.len(),
);
if at == self.len() {
return mem::replace(self, Bytes::new());
@@ -342,7 +414,6 @@ impl Bytes {
return Bytes::new();
}
let mut ret = self.clone();
unsafe { self.inc_start(at) };
@@ -373,10 +444,17 @@ impl Bytes {
/// [`split_off`]: #method.split_off
#[inline]
pub fn truncate(&mut self, len: usize) {
if len >= self.len {
self.len = 0;
} else {
self.len = len;
if len < self.len {
// The Vec "promotable" vtables do not store the capacity,
// so we cannot truncate while using this repr. We *have* to
// promote using `split_off` so the capacity can be stored.
if self.vtable as *const Vtable == &PROMOTABLE_EVEN_VTABLE
|| self.vtable as *const Vtable == &PROMOTABLE_ODD_VTABLE
{
drop(self.split_off(len));
} else {
self.len = len;
}
}
}
@@ -397,7 +475,12 @@ impl Bytes {
}
#[inline]
pub(crate) unsafe fn with_vtable(ptr: *const u8, len: usize, data: AtomicPtr<()>, vtable: &'static Vtable) -> Bytes {
pub(crate) unsafe fn with_vtable(
ptr: *const u8,
len: usize,
data: AtomicPtr<()>,
vtable: &'static Vtable,
) -> Bytes {
Bytes {
ptr,
len,
@@ -410,15 +493,13 @@ impl Bytes {
#[inline]
fn as_slice(&self) -> &[u8] {
unsafe {
slice::from_raw_parts(self.ptr, self.len)
}
unsafe { slice::from_raw_parts(self.ptr, self.len) }
}
#[inline]
unsafe fn inc_start(&mut self, by: usize) {
// should already be asserted, but debug assert for tests
debug_assert!(self.len >= by);
debug_assert!(self.len >= by, "internal: inc_start out of bounds");
self.len -= by;
self.ptr = self.ptr.offset(by as isize);
}
@@ -431,24 +512,14 @@ unsafe impl Sync for Bytes {}
impl Drop for Bytes {
#[inline]
fn drop(&mut self) {
unsafe {
(self.vtable.drop)(&mut self.data, self.ptr, self.len)
}
unsafe { (self.vtable.drop)(&mut self.data, self.ptr, self.len) }
}
}
impl Clone for Bytes {
#[inline]
fn clone(&self) -> Bytes {
unsafe {
(self.vtable.clone)(&self.data, self.ptr, self.len)
}
}
}
impl fmt::Debug for Bytes {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt::Debug::fmt(&debug::BsDebug(&self.as_slice()), f)
unsafe { (self.vtable.clone)(&self.data, self.ptr, self.len) }
}
}
@@ -459,20 +530,32 @@ impl Buf for Bytes {
}
#[inline]
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
self.as_slice()
}
#[inline]
fn advance(&mut self, cnt: usize) {
assert!(cnt <= self.len(), "cannot advance past `remaining`");
assert!(
cnt <= self.len(),
"cannot advance past `remaining`: {:?} <= {:?}",
cnt,
self.len(),
);
unsafe {
self.inc_start(cnt);
}
}
fn to_bytes(&mut self) -> crate::Bytes {
core::mem::replace(self, Bytes::new())
fn copy_to_bytes(&mut self, len: usize) -> crate::Bytes {
if len == self.remaining() {
core::mem::replace(self, Bytes::new())
} else {
let ret = self.slice(..len);
self.advance(len);
ret
}
}
}
@@ -493,7 +576,10 @@ impl AsRef<[u8]> for Bytes {
}
impl hash::Hash for Bytes {
fn hash<H>(&self, state: &mut H) where H: hash::Hasher {
fn hash<H>(&self, state: &mut H)
where
H: hash::Hasher,
{
self.as_slice().hash(state);
}
}
@@ -570,7 +656,7 @@ impl PartialEq<Bytes> for [u8] {
impl PartialOrd<Bytes> for [u8] {
fn partial_cmp(&self, other: &Bytes) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self, other)
}
}
@@ -594,7 +680,7 @@ impl PartialEq<Bytes> for str {
impl PartialOrd<Bytes> for str {
fn partial_cmp(&self, other: &Bytes) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self.as_bytes(), other)
}
}
@@ -618,7 +704,7 @@ impl PartialEq<Bytes> for Vec<u8> {
impl PartialOrd<Bytes> for Vec<u8> {
fn partial_cmp(&self, other: &Bytes) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self, other)
}
}
@@ -642,7 +728,7 @@ impl PartialEq<Bytes> for String {
impl PartialOrd<Bytes> for String {
fn partial_cmp(&self, other: &Bytes) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self.as_bytes(), other)
}
}
@@ -654,7 +740,7 @@ impl PartialEq<Bytes> for &[u8] {
impl PartialOrd<Bytes> for &[u8] {
fn partial_cmp(&self, other: &Bytes) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self, other)
}
}
@@ -666,12 +752,13 @@ impl PartialEq<Bytes> for &str {
impl PartialOrd<Bytes> for &str {
fn partial_cmp(&self, other: &Bytes) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self.as_bytes(), other)
}
}
impl<'a, T: ?Sized> PartialEq<&'a T> for Bytes
where Bytes: PartialEq<T>
where
Bytes: PartialEq<T>,
{
fn eq(&self, other: &&'a T) -> bool {
*self == **other
@@ -679,7 +766,8 @@ impl<'a, T: ?Sized> PartialEq<&'a T> for Bytes
}
impl<'a, T: ?Sized> PartialOrd<&'a T> for Bytes
where Bytes: PartialOrd<T>
where
Bytes: PartialOrd<T>,
{
fn partial_cmp(&self, other: &&'a T) -> Option<cmp::Ordering> {
self.partial_cmp(&**other)
@@ -710,16 +798,37 @@ impl From<&'static str> for Bytes {
impl From<Vec<u8>> for Bytes {
fn from(vec: Vec<u8>) -> Bytes {
let slice = vec.into_boxed_slice();
let len = slice.len();
let ptr = slice.as_ptr();
drop(Box::into_raw(slice));
slice.into()
}
}
let data = ptr as usize | KIND_VEC;
Bytes {
ptr,
len,
data: AtomicPtr::new(data as *mut _),
vtable: &SHARED_VTABLE,
impl From<Box<[u8]>> for Bytes {
fn from(slice: Box<[u8]>) -> Bytes {
// Box<[u8]> doesn't contain a heap allocation for empty slices,
// so the pointer isn't aligned enough for the KIND_VEC stashing to
// work.
if slice.is_empty() {
return Bytes::new();
}
let len = slice.len();
let ptr = Box::into_raw(slice) as *mut u8;
if ptr as usize & 0x1 == 0 {
let data = ptr as usize | KIND_VEC;
Bytes {
ptr,
len,
data: AtomicPtr::new(data as *mut _),
vtable: &PROMOTABLE_EVEN_VTABLE,
}
} else {
Bytes {
ptr,
len,
data: AtomicPtr::new(ptr as *mut _),
vtable: &PROMOTABLE_ODD_VTABLE,
}
}
}
}
@@ -757,6 +866,78 @@ unsafe fn static_drop(_: &mut AtomicPtr<()>, _: *const u8, _: usize) {
// nothing to drop for &'static [u8]
}
// ===== impl PromotableVtable =====
static PROMOTABLE_EVEN_VTABLE: Vtable = Vtable {
clone: promotable_even_clone,
drop: promotable_even_drop,
};
static PROMOTABLE_ODD_VTABLE: Vtable = Vtable {
clone: promotable_odd_clone,
drop: promotable_odd_drop,
};
unsafe fn promotable_even_clone(data: &AtomicPtr<()>, ptr: *const u8, len: usize) -> Bytes {
let shared = data.load(Ordering::Acquire);
let kind = shared as usize & KIND_MASK;
if kind == KIND_ARC {
shallow_clone_arc(shared as _, ptr, len)
} else {
debug_assert_eq!(kind, KIND_VEC);
let buf = (shared as usize & !KIND_MASK) as *mut u8;
shallow_clone_vec(data, shared, buf, ptr, len)
}
}
unsafe fn promotable_even_drop(data: &mut AtomicPtr<()>, ptr: *const u8, len: usize) {
data.with_mut(|shared| {
let shared = *shared;
let kind = shared as usize & KIND_MASK;
if kind == KIND_ARC {
release_shared(shared as *mut Shared);
} else {
debug_assert_eq!(kind, KIND_VEC);
let buf = (shared as usize & !KIND_MASK) as *mut u8;
drop(rebuild_boxed_slice(buf, ptr, len));
}
});
}
unsafe fn promotable_odd_clone(data: &AtomicPtr<()>, ptr: *const u8, len: usize) -> Bytes {
let shared = data.load(Ordering::Acquire);
let kind = shared as usize & KIND_MASK;
if kind == KIND_ARC {
shallow_clone_arc(shared as _, ptr, len)
} else {
debug_assert_eq!(kind, KIND_VEC);
shallow_clone_vec(data, shared, shared as *mut u8, ptr, len)
}
}
unsafe fn promotable_odd_drop(data: &mut AtomicPtr<()>, ptr: *const u8, len: usize) {
data.with_mut(|shared| {
let shared = *shared;
let kind = shared as usize & KIND_MASK;
if kind == KIND_ARC {
release_shared(shared as *mut Shared);
} else {
debug_assert_eq!(kind, KIND_VEC);
drop(rebuild_boxed_slice(shared as *mut u8, ptr, len));
}
});
}
unsafe fn rebuild_boxed_slice(buf: *mut u8, offset: *const u8, len: usize) -> Box<[u8]> {
let cap = (offset as usize - buf as usize) + len;
Box::from_raw(slice::from_raw_parts_mut(buf, cap))
}
// ===== impl SharedVtable =====
struct Shared {
@@ -765,6 +946,12 @@ struct Shared {
ref_cnt: AtomicUsize,
}
// Assert that the alignment of `Shared` is divisible by 2.
// This is a necessary invariant since we depend on allocating `Shared` a
// shared object to implicitly carry the `KIND_ARC` flag in its pointer.
// This flag is set when the LSB is 0.
const _: [(); 0 - mem::align_of::<Shared>() % 2] = []; // Assert that the alignment of `Shared` is divisible by 2.
static SHARED_VTABLE: Vtable = Vtable {
clone: shared_clone,
drop: shared_drop,
@@ -775,37 +962,14 @@ const KIND_VEC: usize = 0b1;
const KIND_MASK: usize = 0b1;
unsafe fn shared_clone(data: &AtomicPtr<()>, ptr: *const u8, len: usize) -> Bytes {
let shared = data.load(Ordering::Acquire);
let kind = shared as usize & KIND_MASK;
if kind == KIND_ARC {
shallow_clone_arc(shared as _, ptr, len)
} else {
debug_assert_eq!(kind, KIND_VEC);
shallow_clone_vec(data, shared, ptr, len)
}
let shared = data.load(Ordering::Relaxed);
shallow_clone_arc(shared as _, ptr, len)
}
unsafe fn shared_drop(data: &mut AtomicPtr<()>, ptr: *const u8, len: usize) {
let shared = *data.get_mut();
let kind = shared as usize & KIND_MASK;
if kind == KIND_ARC {
release_shared(shared as *mut Shared);
} else {
debug_assert_eq!(kind, KIND_VEC);
drop(rebuild_vec(shared, ptr, len));
}
}
unsafe fn rebuild_vec(shared: *const (), offset: *const u8, len: usize) -> Vec<u8> {
debug_assert_eq!(shared as usize & KIND_MASK, KIND_VEC);
let buf = (shared as usize & !KIND_MASK) as *mut u8;
let cap = (offset as usize - buf as usize) + len;
Vec::from_raw_parts(buf, cap, cap)
unsafe fn shared_drop(data: &mut AtomicPtr<()>, _ptr: *const u8, _len: usize) {
data.with_mut(|shared| {
release_shared(*shared as *mut Shared);
});
}
unsafe fn shallow_clone_arc(shared: *mut Shared, ptr: *const u8, len: usize) -> Bytes {
@@ -824,13 +988,17 @@ unsafe fn shallow_clone_arc(shared: *mut Shared, ptr: *const u8, len: usize) ->
}
#[cold]
unsafe fn shallow_clone_vec(atom: &AtomicPtr<()>, ptr: *const (), offset: *const u8, len: usize) -> Bytes {
unsafe fn shallow_clone_vec(
atom: &AtomicPtr<()>,
ptr: *const (),
buf: *mut u8,
offset: *const u8,
len: usize,
) -> Bytes {
// If the buffer is still tracked in a `Vec<u8>`. It is time to
// promote the vec to an `Arc`. This could potentially be called
// concurrently, so some care must be taken.
debug_assert_eq!(ptr as usize & KIND_MASK, KIND_VEC);
// First, allocate a new `Shared` instance containing the
// `Vec` fields. It's important to note that `ptr`, `len`,
// and `cap` cannot be mutated without having `&mut self`.
@@ -838,7 +1006,7 @@ unsafe fn shallow_clone_vec(atom: &AtomicPtr<()>, ptr: *const (), offset: *const
// updated and since the buffer hasn't been promoted to an
// `Arc`, those three fields still are the components of the
// vector.
let vec = rebuild_vec(ptr as *const (), offset, len);
let vec = rebuild_boxed_slice(buf, offset, len).into_vec();
let shared = Box::new(Shared {
_vec: vec,
// Initialize refcount to 2. One for this reference, and one
@@ -851,39 +1019,44 @@ unsafe fn shallow_clone_vec(atom: &AtomicPtr<()>, ptr: *const (), offset: *const
// The pointer should be aligned, so this assert should
// always succeed.
debug_assert!(0 == (shared as usize & KIND_MASK));
debug_assert!(
0 == (shared as usize & KIND_MASK),
"internal: Box<Shared> should have an aligned pointer",
);
// Try compare & swapping the pointer into the `arc` field.
// `Release` is used synchronize with other threads that
// will load the `arc` field.
//
// If the `compare_and_swap` fails, then the thread lost the
// If the `compare_exchange` fails, then the thread lost the
// race to promote the buffer to shared. The `Acquire`
// ordering will synchronize with the `compare_and_swap`
// ordering will synchronize with the `compare_exchange`
// that happened in the other thread and the `Shared`
// pointed to by `actual` will be visible.
let actual = atom.compare_and_swap(ptr as _, shared as _, Ordering::AcqRel);
match atom.compare_exchange(ptr as _, shared as _, Ordering::AcqRel, Ordering::Acquire) {
Ok(actual) => {
debug_assert!(actual as usize == ptr as usize);
// The upgrade was successful, the new handle can be
// returned.
Bytes {
ptr: offset,
len,
data: AtomicPtr::new(shared as _),
vtable: &SHARED_VTABLE,
}
}
Err(actual) => {
// The upgrade failed, a concurrent clone happened. Release
// the allocation that was made in this thread, it will not
// be needed.
let shared = Box::from_raw(shared);
mem::forget(*shared);
if actual as usize == ptr as usize {
// The upgrade was successful, the new handle can be
// returned.
return Bytes {
ptr: offset,
len,
data: AtomicPtr::new(shared as _),
vtable: &SHARED_VTABLE,
};
// Buffer already promoted to shared storage, so increment ref
// count.
shallow_clone_arc(actual as _, offset, len)
}
}
// The upgrade failed, a concurrent clone happened. Release
// the allocation that was made in this thread, it will not
// be needed.
let shared = Box::from_raw(shared);
mem::forget(*shared);
// Buffer already promoted to shared storage, so increment ref
// count.
shallow_clone_arc(actual as _, offset, len)
}
unsafe fn release_shared(ptr: *mut Shared) {
@@ -915,10 +1088,32 @@ unsafe fn release_shared(ptr: *mut Shared) {
Box::from_raw(ptr);
}
// compile-fails
/// ```compile_fail
/// use bytes::Bytes;
/// #[deny(unused_must_use)]
/// {
/// let mut b1 = Bytes::from("hello world");
/// b1.split_to(6);
/// }
/// ```
fn _split_to_must_use() {}
/// ```compile_fail
/// use bytes::Bytes;
/// #[deny(unused_must_use)]
/// {
/// let mut b1 = Bytes::from("hello world");
/// b1.split_off(6);
/// }
/// ```
fn _split_off_must_use() {}
// fuzz tests
#[cfg(all(test, loom))]
mod fuzz {
use std::sync::Arc;
use loom::sync::Arc;
use loom::thread;
use super::Bytes;
+221 -87
View File
@@ -1,23 +1,33 @@
use core::{cmp, fmt, hash, isize, slice, usize};
use core::iter::{FromIterator, Iterator};
use core::mem::{self, ManuallyDrop};
use core::ops::{Deref, DerefMut};
use core::ptr::{self, NonNull};
use core::iter::{FromIterator, Iterator};
use core::{cmp, fmt, hash, isize, slice, usize};
use alloc::{vec::Vec, string::String, boxed::Box, borrow::{Borrow, BorrowMut}};
use alloc::{
borrow::{Borrow, BorrowMut},
boxed::Box,
string::String,
vec::Vec,
};
use crate::{Bytes, Buf, BufMut};
use crate::buf::{IntoIter, UninitSlice};
use crate::bytes::Vtable;
use crate::buf::IntoIter;
use crate::debug;
#[allow(unused)]
use crate::loom::sync::atomic::AtomicMut;
use crate::loom::sync::atomic::{self, AtomicPtr, AtomicUsize, Ordering};
use crate::{Buf, BufMut, Bytes};
/// A unique reference to a contiguous slice of memory.
///
/// `BytesMut` represents a unique view into a potentially shared memory region.
/// Given the uniqueness guarantee, owners of `BytesMut` handles are able to
/// mutate the memory. It is similar to a `Vec<u8>` but with less copies and
/// allocations.
/// mutate the memory.
///
/// `BytesMut` can be thought of as containing a `buf: Arc<Vec<u8>>`, an offset
/// into `buf`, a slice length, and a guarantee that no other `BytesMut` for the
/// same `buf` overlaps with its slice. That guarantee means that a write lock
/// is not required.
///
/// # Growth
///
@@ -108,8 +118,7 @@ impl BytesMut {
/// Creates a new `BytesMut` with the specified capacity.
///
/// The returned `BytesMut` will be able to hold at least `capacity` bytes
/// without reallocating. If `capacity` is under `4 * size_of::<usize>() - 1`,
/// then `BytesMut` will not allocate.
/// without reallocating.
///
/// It is important to note that this function does not specify the length
/// of the returned `BytesMut`, but only the capacity.
@@ -234,7 +243,9 @@ impl BytesMut {
let (off, _) = self.get_vec_pos();
let vec = rebuild_vec(self.ptr.as_ptr(), self.len, self.cap, off);
mem::forget(self);
vec.into()
let mut b: Bytes = vec.into();
b.advance(off);
b
}
} else {
debug_assert_eq!(self.kind(), KIND_ARC);
@@ -243,9 +254,7 @@ impl BytesMut {
let len = self.len;
let data = AtomicPtr::new(self.data as _);
mem::forget(self);
unsafe {
Bytes::with_vtable(ptr, len, data, &SHARED_VTABLE)
}
unsafe { Bytes::with_vtable(ptr, len, data, &SHARED_VTABLE) }
}
}
@@ -275,8 +284,14 @@ impl BytesMut {
/// # Panics
///
/// Panics if `at > capacity`.
#[must_use = "consider BytesMut::truncate if you don't need the other half"]
pub fn split_off(&mut self, at: usize) -> BytesMut {
assert!(at <= self.capacity());
assert!(
at <= self.capacity(),
"split_off out of bounds: {:?} <= {:?}",
at,
self.capacity(),
);
unsafe {
let mut other = self.shallow_clone();
other.set_start(at);
@@ -310,6 +325,7 @@ impl BytesMut {
///
/// assert_eq!(other, b"hello world"[..]);
/// ```
#[must_use = "consider BytesMut::advance(len()) if you don't need the other half"]
pub fn split(&mut self) -> BytesMut {
let len = self.len();
self.split_to(len)
@@ -341,8 +357,14 @@ impl BytesMut {
/// # Panics
///
/// Panics if `at > len`.
#[must_use = "consider BytesMut::advance if you don't need the other half"]
pub fn split_to(&mut self, at: usize) -> BytesMut {
assert!(at <= self.len());
assert!(
at <= self.len(),
"split_to out of bounds: {:?} <= {:?}",
at,
self.len(),
);
unsafe {
let mut other = self.shallow_clone();
@@ -358,6 +380,8 @@ impl BytesMut {
/// If `len` is greater than the buffer's current length, this has no
/// effect.
///
/// Existing underlying capacity is preserved.
///
/// The [`split_off`] method can emulate `truncate`, but this causes the
/// excess bytes to be returned instead of dropped.
///
@@ -374,11 +398,13 @@ impl BytesMut {
/// [`split_off`]: #method.split_off
pub fn truncate(&mut self, len: usize) {
if len <= self.len() {
unsafe { self.set_len(len); }
unsafe {
self.set_len(len);
}
}
}
/// Clears the buffer, removing all data.
/// Clears the buffer, removing all data. Existing capacity is preserved.
///
/// # Examples
///
@@ -421,7 +447,7 @@ impl BytesMut {
let additional = new_len - len;
self.reserve(additional);
unsafe {
let dst = self.bytes_mut().as_mut_ptr();
let dst = self.chunk_mut().as_mut_ptr();
ptr::write_bytes(dst, value, additional);
self.set_len(new_len);
}
@@ -455,8 +481,9 @@ impl BytesMut {
///
/// assert_eq!(&b[..], b"hello world");
/// ```
#[inline]
pub unsafe fn set_len(&mut self, len: usize) {
debug_assert!(len <= self.cap);
debug_assert!(len <= self.cap, "set_len out of bounds");
self.len = len;
}
@@ -538,9 +565,8 @@ impl BytesMut {
unsafe {
let (off, prev) = self.get_vec_pos();
// Only reuse space if we stand to gain at least capacity/2
// bytes of space back
if off >= additional && off >= (self.cap / 2) {
// Only reuse space if we can satisfy the requested additional space.
if self.capacity() - self.len() + off >= additional {
// There's space - reuse it
//
// Just move the pointer back to the start after copying
@@ -555,7 +581,8 @@ impl BytesMut {
self.cap += off;
} else {
// No space - allocate more
let mut v = ManuallyDrop::new(rebuild_vec(self.ptr.as_ptr(), self.len, self.cap, off));
let mut v =
ManuallyDrop::new(rebuild_vec(self.ptr.as_ptr(), self.len, self.cap, off));
v.reserve(additional);
// Update the info
@@ -571,7 +598,6 @@ impl BytesMut {
debug_assert_eq!(kind, KIND_ARC);
let shared: *mut Shared = self.data as _;
// Reserving involves abandoning the currently shared buffer and
// allocating a new vector with the requested capacity.
//
@@ -615,9 +641,7 @@ impl BytesMut {
// check.
let double = v.capacity().checked_shl(1).unwrap_or(new_cap);
new_cap = cmp::max(
cmp::max(double, new_cap),
original_capacity);
new_cap = cmp::max(cmp::max(double, new_cap), original_capacity);
} else {
new_cap = cmp::max(new_cap, original_capacity);
}
@@ -640,10 +664,11 @@ impl BytesMut {
self.len = v.len();
self.cap = v.capacity();
}
/// Appends given bytes to this object.
/// Appends given bytes to this `BytesMut`.
///
/// If this `BytesMut` object has not enough capacity, it is resized first.
/// So unlike `put_slice` operation, `extend_from_slice` does not panic.
/// If this `BytesMut` object does not have enough capacity, it is resized
/// first.
///
/// # Examples
///
@@ -657,13 +682,29 @@ impl BytesMut {
/// assert_eq!(b"aaabbbcccddd", &buf[..]);
/// ```
pub fn extend_from_slice(&mut self, extend: &[u8]) {
self.reserve(extend.len());
self.put_slice(extend);
let cnt = extend.len();
self.reserve(cnt);
unsafe {
let dst = self.uninit_slice();
// Reserved above
debug_assert!(dst.len() >= cnt);
ptr::copy_nonoverlapping(extend.as_ptr(), dst.as_mut_ptr() as *mut u8, cnt);
}
unsafe {
self.advance_mut(cnt);
}
}
/// Combine splitted BytesMut objects back as contiguous.
/// Absorbs a `BytesMut` that was previously split off.
///
/// If `BytesMut` objects were not contiguous originally, they will be extended.
/// If the two `BytesMut` objects were previously contiguous, i.e., if
/// `other` was created by calling `split_off` on this `BytesMut`, then
/// this is an `O(1)` operation that just decreases a reference
/// count and sets a few indices. Otherwise this method degenerates to
/// `self.extend_from_slice(other.as_ref())`.
///
/// # Examples
///
@@ -673,11 +714,11 @@ impl BytesMut {
/// let mut buf = BytesMut::with_capacity(64);
/// buf.extend_from_slice(b"aaabbbcccddd");
///
/// let splitted = buf.split_off(6);
/// let split = buf.split_off(6);
/// assert_eq!(b"aaabbb", &buf[..]);
/// assert_eq!(b"cccddd", &splitted[..]);
/// assert_eq!(b"cccddd", &split[..]);
///
/// buf.unsplit(splitted);
/// buf.unsplit(split);
/// assert_eq!(b"aaabbbcccddd", &buf[..]);
/// ```
pub fn unsplit(&mut self, other: BytesMut) {
@@ -719,16 +760,12 @@ impl BytesMut {
#[inline]
fn as_slice(&self) -> &[u8] {
unsafe {
slice::from_raw_parts(self.ptr.as_ptr(), self.len)
}
unsafe { slice::from_raw_parts(self.ptr.as_ptr(), self.len) }
}
#[inline]
fn as_slice_mut(&mut self) -> &mut [u8] {
unsafe {
slice::from_raw_parts_mut(self.ptr.as_ptr(), self.len)
}
unsafe { slice::from_raw_parts_mut(self.ptr.as_ptr(), self.len) }
}
unsafe fn set_start(&mut self, start: usize) {
@@ -738,7 +775,7 @@ impl BytesMut {
return;
}
debug_assert!(start <= self.cap);
debug_assert!(start <= self.cap, "internal: set_start out of bounds");
let kind = self.kind();
@@ -757,7 +794,7 @@ impl BytesMut {
// on 64 bit systems and will only happen on 32 bit systems
// when shifting past 134,217,727 bytes. As such, we don't
// worry too much about performance here.
self.promote_to_shared(/*ref_count = */1);
self.promote_to_shared(/*ref_count = */ 1);
}
}
@@ -777,22 +814,22 @@ impl BytesMut {
unsafe fn set_end(&mut self, end: usize) {
debug_assert_eq!(self.kind(), KIND_ARC);
assert!(end <= self.cap);
assert!(end <= self.cap, "set_end out of bounds");
self.cap = end;
self.len = cmp::min(self.len, end);
}
fn try_unsplit(&mut self, other: BytesMut) -> Result<(), BytesMut> {
if other.is_empty() {
if other.capacity() == 0 {
return Ok(());
}
let ptr = unsafe { self.ptr.as_ptr().offset(self.len as isize) };
if ptr == other.ptr.as_ptr() &&
self.kind() == KIND_ARC &&
other.kind() == KIND_ARC &&
self.data == other.data
if ptr == other.ptr.as_ptr()
&& self.kind() == KIND_ARC
&& other.kind() == KIND_ARC
&& self.data == other.data
{
// Contiguous blocks, just combine directly
self.len += other.len;
@@ -853,7 +890,7 @@ impl BytesMut {
increment_shared(self.data);
ptr::read(self)
} else {
self.promote_to_shared(/*ref_count = */2);
self.promote_to_shared(/*ref_count = */ 2);
ptr::read(self)
}
}
@@ -873,6 +910,16 @@ impl BytesMut {
self.data = ((pos << VEC_POS_OFFSET) | (prev & NOT_VEC_POS_MASK)) as *mut _;
}
#[inline]
fn uninit_slice(&mut self) -> &mut UninitSlice {
unsafe {
let ptr = self.ptr.as_ptr().offset(self.len as isize);
let len = self.cap - self.len;
UninitSlice::from_raw_parts_mut(ptr, len)
}
}
}
impl Drop for BytesMut {
@@ -899,22 +946,29 @@ impl Buf for BytesMut {
}
#[inline]
fn bytes(&self) -> &[u8] {
fn chunk(&self) -> &[u8] {
self.as_slice()
}
#[inline]
fn advance(&mut self, cnt: usize) {
assert!(cnt <= self.remaining(), "cannot advance past `remaining`");
unsafe { self.set_start(cnt); }
assert!(
cnt <= self.remaining(),
"cannot advance past `remaining`: {:?} <= {:?}",
cnt,
self.remaining(),
);
unsafe {
self.set_start(cnt);
}
}
fn to_bytes(&mut self) -> crate::Bytes {
self.split().freeze()
fn copy_to_bytes(&mut self, len: usize) -> crate::Bytes {
self.split_to(len).freeze()
}
}
impl BufMut for BytesMut {
unsafe impl BufMut for BytesMut {
#[inline]
fn remaining_mut(&self) -> usize {
usize::MAX - self.len()
@@ -923,21 +977,52 @@ impl BufMut for BytesMut {
#[inline]
unsafe fn advance_mut(&mut self, cnt: usize) {
let new_len = self.len() + cnt;
assert!(new_len <= self.cap, "new_len = {}; capacity = {}", new_len, self.cap);
assert!(
new_len <= self.cap,
"new_len = {}; capacity = {}",
new_len,
self.cap
);
self.len = new_len;
}
#[inline]
fn bytes_mut(&mut self) -> &mut [mem::MaybeUninit<u8>] {
fn chunk_mut(&mut self) -> &mut UninitSlice {
if self.capacity() == self.len() {
self.reserve(64);
}
self.uninit_slice()
}
// Specialize these methods so they can skip checking `remaining_mut`
// and `advance_mut`.
fn put<T: crate::Buf>(&mut self, mut src: T)
where
Self: Sized,
{
while src.has_remaining() {
let s = src.chunk();
let l = s.len();
self.extend_from_slice(s);
src.advance(l);
}
}
fn put_slice(&mut self, src: &[u8]) {
self.extend_from_slice(src);
}
fn put_bytes(&mut self, val: u8, cnt: usize) {
self.reserve(cnt);
unsafe {
let ptr = self.ptr.as_ptr().offset(self.len as isize);
let len = self.cap - self.len;
let dst = self.uninit_slice();
// Reserved above
debug_assert!(dst.len() >= cnt);
slice::from_raw_parts_mut(ptr as *mut mem::MaybeUninit<u8>, len)
ptr::write_bytes(dst.as_mut_ptr(), val, cnt);
self.advance_mut(cnt);
}
}
}
@@ -959,6 +1044,7 @@ impl Deref for BytesMut {
}
impl AsMut<[u8]> for BytesMut {
#[inline]
fn as_mut(&mut self) -> &mut [u8] {
self.as_slice_mut()
}
@@ -983,6 +1069,12 @@ impl<'a> From<&'a str> for BytesMut {
}
}
impl From<BytesMut> for Bytes {
fn from(src: BytesMut) -> Bytes {
src.freeze()
}
}
impl PartialEq for BytesMut {
fn eq(&self, other: &BytesMut) -> bool {
self.as_slice() == other.as_slice()
@@ -1001,8 +1093,7 @@ impl Ord for BytesMut {
}
}
impl Eq for BytesMut {
}
impl Eq for BytesMut {}
impl Default for BytesMut {
#[inline]
@@ -1011,14 +1102,11 @@ impl Default for BytesMut {
}
}
impl fmt::Debug for BytesMut {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt::Debug::fmt(&debug::BsDebug(&self.as_slice()), fmt)
}
}
impl hash::Hash for BytesMut {
fn hash<H>(&self, state: &mut H) where H: hash::Hasher {
fn hash<H>(&self, state: &mut H)
where
H: hash::Hasher,
{
let s: &[u8] = self.as_ref();
s.hash(state);
}
@@ -1078,7 +1166,10 @@ impl<'a> IntoIterator for &'a BytesMut {
}
impl Extend<u8> for BytesMut {
fn extend<T>(&mut self, iter: T) where T: IntoIterator<Item = u8> {
fn extend<T>(&mut self, iter: T)
where
T: IntoIterator<Item = u8>,
{
let iter = iter.into_iter();
let (lower, _) = iter.size_hint();
@@ -1095,7 +1186,10 @@ impl Extend<u8> for BytesMut {
}
impl<'a> Extend<&'a u8> for BytesMut {
fn extend<T>(&mut self, iter: T) where T: IntoIterator<Item = &'a u8> {
fn extend<T>(&mut self, iter: T)
where
T: IntoIterator<Item = &'a u8>,
{
self.extend(iter.into_iter().map(|b| *b))
}
}
@@ -1171,9 +1265,13 @@ impl Shared {
}
}
#[inline]
fn original_capacity_to_repr(cap: usize) -> usize {
let width = PTR_WIDTH - ((cap >> MIN_ORIGINAL_CAPACITY_WIDTH).leading_zeros() as usize);
cmp::min(width, MAX_ORIGINAL_CAPACITY_WIDTH - MIN_ORIGINAL_CAPACITY_WIDTH)
cmp::min(
width,
MAX_ORIGINAL_CAPACITY_WIDTH - MIN_ORIGINAL_CAPACITY_WIDTH,
)
}
fn original_capacity_from_repr(repr: usize) -> usize {
@@ -1264,7 +1362,7 @@ impl PartialEq<BytesMut> for [u8] {
impl PartialOrd<BytesMut> for [u8] {
fn partial_cmp(&self, other: &BytesMut) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self, other)
}
}
@@ -1288,7 +1386,7 @@ impl PartialEq<BytesMut> for str {
impl PartialOrd<BytesMut> for str {
fn partial_cmp(&self, other: &BytesMut) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self.as_bytes(), other)
}
}
@@ -1336,12 +1434,13 @@ impl PartialEq<BytesMut> for String {
impl PartialOrd<BytesMut> for String {
fn partial_cmp(&self, other: &BytesMut) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self.as_bytes(), other)
}
}
impl<'a, T: ?Sized> PartialEq<&'a T> for BytesMut
where BytesMut: PartialEq<T>
where
BytesMut: PartialEq<T>,
{
fn eq(&self, other: &&'a T) -> bool {
*self == **other
@@ -1349,7 +1448,8 @@ impl<'a, T: ?Sized> PartialEq<&'a T> for BytesMut
}
impl<'a, T: ?Sized> PartialOrd<&'a T> for BytesMut
where BytesMut: PartialOrd<T>
where
BytesMut: PartialOrd<T>,
{
fn partial_cmp(&self, other: &&'a T) -> Option<cmp::Ordering> {
self.partial_cmp(*other)
@@ -1364,7 +1464,7 @@ impl PartialEq<BytesMut> for &[u8] {
impl PartialOrd<BytesMut> for &[u8] {
fn partial_cmp(&self, other: &BytesMut) -> Option<cmp::Ordering> {
other.partial_cmp(self)
<[u8] as PartialOrd<[u8]>>::partial_cmp(self, other)
}
}
@@ -1392,6 +1492,7 @@ impl PartialEq<Bytes> for BytesMut {
}
}
#[inline]
fn vptr(ptr: *mut u8) -> NonNull<u8> {
if cfg!(debug_assertions) {
NonNull::new(ptr).expect("Vec pointer should be non-null")
@@ -1416,7 +1517,7 @@ static SHARED_VTABLE: Vtable = Vtable {
};
unsafe fn shared_v_clone(data: &AtomicPtr<()>, ptr: *const u8, len: usize) -> Bytes {
let shared = data.load(Ordering::Acquire) as *mut Shared;
let shared = data.load(Ordering::Relaxed) as *mut Shared;
increment_shared(shared);
let data = AtomicPtr::new(shared as _);
@@ -1424,18 +1525,51 @@ unsafe fn shared_v_clone(data: &AtomicPtr<()>, ptr: *const u8, len: usize) -> By
}
unsafe fn shared_v_drop(data: &mut AtomicPtr<()>, _ptr: *const u8, _len: usize) {
let shared = (*data.get_mut()) as *mut Shared;
release_shared(shared as *mut Shared);
data.with_mut(|shared| {
release_shared(*shared as *mut Shared);
});
}
// compile-fails
/// ```compile_fail
/// use bytes::BytesMut;
/// #[deny(unused_must_use)]
/// {
/// let mut b1 = BytesMut::from("hello world");
/// b1.split_to(6);
/// }
/// ```
fn _split_to_must_use() {}
/// ```compile_fail
/// use bytes::BytesMut;
/// #[deny(unused_must_use)]
/// {
/// let mut b1 = BytesMut::from("hello world");
/// b1.split_off(6);
/// }
/// ```
fn _split_off_must_use() {}
/// ```compile_fail
/// use bytes::BytesMut;
/// #[deny(unused_must_use)]
/// {
/// let mut b1 = BytesMut::from("hello world");
/// b1.split();
/// }
/// ```
fn _split_must_use() {}
// fuzz tests
#[cfg(all(test, loom))]
mod fuzz {
use std::sync::Arc;
use loom::sync::Arc;
use loom::thread;
use crate::Bytes;
use super::BytesMut;
use crate::Bytes;
#[test]
fn bytes_mut_cloning_frozen() {
-40
View File
@@ -1,40 +0,0 @@
use core::fmt;
/// Alternative implementation of `fmt::Debug` for byte slice.
///
/// Standard `Debug` implementation for `[u8]` is comma separated
/// list of numbers. Since large amount of byte strings are in fact
/// ASCII strings or contain a lot of ASCII strings (e. g. HTTP),
/// it is convenient to print strings as ASCII when possible.
///
/// This struct wraps `&[u8]` just to override `fmt::Debug`.
///
/// `BsDebug` is not a part of public API of bytes crate.
pub struct BsDebug<'a>(pub &'a [u8]);
impl fmt::Debug for BsDebug<'_> {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> Result<(), fmt::Error> {
write!(fmt, "b\"")?;
for &c in self.0 {
// https://doc.rust-lang.org/reference.html#byte-escapes
if c == b'\n' {
write!(fmt, "\\n")?;
} else if c == b'\r' {
write!(fmt, "\\r")?;
} else if c == b'\t' {
write!(fmt, "\\t")?;
} else if c == b'\\' || c == b'"' {
write!(fmt, "\\{}", c as char)?;
} else if c == b'\0' {
write!(fmt, "\\0")?;
// ASCII printable
} else if c >= 0x20 && c < 0x7f {
write!(fmt, "{}", c as char)?;
} else {
write!(fmt, "\\x{:02x}", c)?;
}
}
write!(fmt, "\"")?;
Ok(())
}
}
+49
View File
@@ -0,0 +1,49 @@
use core::fmt::{Debug, Formatter, Result};
use super::BytesRef;
use crate::{Bytes, BytesMut};
/// Alternative implementation of `std::fmt::Debug` for byte slice.
///
/// Standard `Debug` implementation for `[u8]` is comma separated
/// list of numbers. Since large amount of byte strings are in fact
/// ASCII strings or contain a lot of ASCII strings (e. g. HTTP),
/// it is convenient to print strings as ASCII when possible.
impl Debug for BytesRef<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
write!(f, "b\"")?;
for &b in self.0 {
// https://doc.rust-lang.org/reference/tokens.html#byte-escapes
if b == b'\n' {
write!(f, "\\n")?;
} else if b == b'\r' {
write!(f, "\\r")?;
} else if b == b'\t' {
write!(f, "\\t")?;
} else if b == b'\\' || b == b'"' {
write!(f, "\\{}", b as char)?;
} else if b == b'\0' {
write!(f, "\\0")?;
// ASCII printable
} else if b >= 0x20 && b < 0x7f {
write!(f, "{}", b as char)?;
} else {
write!(f, "\\x{:02x}", b)?;
}
}
write!(f, "\"")?;
Ok(())
}
}
impl Debug for Bytes {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Debug::fmt(&BytesRef(&self.as_ref()), f)
}
}
impl Debug for BytesMut {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
Debug::fmt(&BytesRef(&self.as_ref()), f)
}
}
+6 -6
View File
@@ -1,20 +1,20 @@
use crate::{Bytes, BytesMut};
use core::fmt::{Formatter, LowerHex, Result, UpperHex};
struct BytesRef<'a>(&'a [u8]);
use super::BytesRef;
use crate::{Bytes, BytesMut};
impl<'a> LowerHex for BytesRef<'a> {
impl LowerHex for BytesRef<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
for b in self.0 {
for &b in self.0 {
write!(f, "{:02x}", b)?;
}
Ok(())
}
}
impl<'a> UpperHex for BytesRef<'a> {
impl UpperHex for BytesRef<'_> {
fn fmt(&self, f: &mut Formatter<'_>) -> Result {
for b in self.0 {
for &b in self.0 {
write!(f, "{:02X}", b)?;
}
Ok(())
+5
View File
@@ -0,0 +1,5 @@
mod debug;
mod hex;
/// `BytesRef` is not a part of public API of bytes crate.
struct BytesRef<'a>(&'a [u8]);
+9 -11
View File
@@ -1,5 +1,8 @@
#![deny(warnings, missing_docs, missing_debug_implementations, rust_2018_idioms)]
#![doc(html_root_url = "https://docs.rs/bytes/0.5.2")]
#![warn(missing_docs, missing_debug_implementations, rust_2018_idioms)]
#![doc(test(
no_crate_inject,
attr(deny(warnings, rust_2018_idioms), allow(dead_code, unused_variables))
))]
#![no_std]
//! Provides abstractions for working with bytes.
@@ -72,25 +75,20 @@
//! perform a syscall, which has the potential of failing. Operations on `Buf`
//! and `BufMut` are infallible.
extern crate alloc;
#[cfg(feature = "std")]
extern crate std;
pub mod buf;
pub use crate::buf::{
Buf,
BufMut,
};
pub use crate::buf::{Buf, BufMut};
mod bytes_mut;
mod bytes;
mod debug;
mod hex;
mod bytes_mut;
mod fmt;
mod loom;
pub use crate::bytes_mut::BytesMut;
pub use crate::bytes::Bytes;
pub use crate::bytes_mut::BytesMut;
// Optional Serde support
#[cfg(feature = "serde")]
+22 -1
View File
@@ -2,8 +2,29 @@
pub(crate) mod sync {
pub(crate) mod atomic {
pub(crate) use core::sync::atomic::{fence, AtomicPtr, AtomicUsize, Ordering};
pub(crate) trait AtomicMut<T> {
fn with_mut<F, R>(&mut self, f: F) -> R
where
F: FnOnce(&mut *mut T) -> R;
}
impl<T> AtomicMut<T> for AtomicPtr<T> {
fn with_mut<F, R>(&mut self, f: F) -> R
where
F: FnOnce(&mut *mut T) -> R,
{
f(self.get_mut())
}
}
}
}
#[cfg(all(test, loom))]
pub(crate) use ::loom::sync;
pub(crate) mod sync {
pub(crate) mod atomic {
pub(crate) use loom::sync::atomic::{fence, AtomicPtr, AtomicUsize, Ordering};
pub(crate) trait AtomicMut<T> {}
}
}
+18 -11
View File
@@ -1,15 +1,16 @@
use super::{Bytes, BytesMut};
use alloc::string::String;
use alloc::vec::Vec;
use core::{cmp, fmt};
use serde::{Serialize, Serializer, Deserialize, Deserializer, de};
use super::{Bytes, BytesMut};
use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
macro_rules! serde_impl {
($ty:ident, $visitor_ty:ident, $from_slice:ident, $from_vec:ident) => (
($ty:ident, $visitor_ty:ident, $from_slice:ident, $from_vec:ident) => {
impl Serialize for $ty {
#[inline]
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where S: Serializer
where
S: Serializer,
{
serializer.serialize_bytes(&self)
}
@@ -26,7 +27,8 @@ macro_rules! serde_impl {
#[inline]
fn visit_seq<V>(self, mut seq: V) -> Result<Self::Value, V::Error>
where V: de::SeqAccess<'de>
where
V: de::SeqAccess<'de>,
{
let len = cmp::min(seq.size_hint().unwrap_or(0), 4096);
let mut values: Vec<u8> = Vec::with_capacity(len);
@@ -40,28 +42,32 @@ macro_rules! serde_impl {
#[inline]
fn visit_bytes<E>(self, v: &[u8]) -> Result<Self::Value, E>
where E: de::Error
where
E: de::Error,
{
Ok($ty::$from_slice(v))
}
#[inline]
fn visit_byte_buf<E>(self, v: Vec<u8>) -> Result<Self::Value, E>
where E: de::Error
where
E: de::Error,
{
Ok($ty::$from_vec(v))
}
#[inline]
fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
where E: de::Error
where
E: de::Error,
{
Ok($ty::$from_slice(v.as_bytes()))
}
#[inline]
fn visit_string<E>(self, v: String) -> Result<Self::Value, E>
where E: de::Error
where
E: de::Error,
{
Ok($ty::$from_vec(v.into_bytes()))
}
@@ -70,12 +76,13 @@ macro_rules! serde_impl {
impl<'de> Deserialize<'de> for $ty {
#[inline]
fn deserialize<D>(deserializer: D) -> Result<$ty, D::Error>
where D: Deserializer<'de>
where
D: Deserializer<'de>,
{
deserializer.deserialize_byte_buf($visitor_ty)
}
}
);
};
}
serde_impl!(Bytes, BytesVisitor, copy_from_slice, from);
+56 -7
View File
@@ -1,6 +1,7 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::Buf;
#[cfg(feature = "std")]
use std::io::IoSlice;
#[test]
@@ -8,17 +9,17 @@ fn test_fresh_cursor_vec() {
let mut buf = &b"hello"[..];
assert_eq!(buf.remaining(), 5);
assert_eq!(buf.bytes(), b"hello");
assert_eq!(buf.chunk(), b"hello");
buf.advance(2);
assert_eq!(buf.remaining(), 3);
assert_eq!(buf.bytes(), b"llo");
assert_eq!(buf.chunk(), b"llo");
buf.advance(3);
assert_eq!(buf.remaining(), 0);
assert_eq!(buf.bytes(), b"");
assert_eq!(buf.chunk(), b"");
}
#[test]
@@ -42,6 +43,7 @@ fn test_get_u16_buffer_underflow() {
buf.get_u16();
}
#[cfg(feature = "std")]
#[test]
fn test_bufs_vec() {
let buf = &b"hello world"[..];
@@ -51,7 +53,7 @@ fn test_bufs_vec() {
let mut dst = [IoSlice::new(b1), IoSlice::new(b2)];
assert_eq!(1, buf.bytes_vectored(&mut dst[..]));
assert_eq!(1, buf.chunks_vectored(&mut dst[..]));
}
#[test]
@@ -61,11 +63,58 @@ fn test_vec_deque() {
let mut buffer: VecDeque<u8> = VecDeque::new();
buffer.extend(b"hello world");
assert_eq!(11, buffer.remaining());
assert_eq!(b"hello world", buffer.bytes());
assert_eq!(b"hello world", buffer.chunk());
buffer.advance(6);
assert_eq!(b"world", buffer.bytes());
assert_eq!(b"world", buffer.chunk());
buffer.extend(b" piece");
let mut out = [0; 11];
buffer.copy_to_slice(&mut out);
assert_eq!(b"world piece", &out[..]);
}
#[test]
fn test_deref_buf_forwards() {
struct Special;
impl Buf for Special {
fn remaining(&self) -> usize {
unreachable!("remaining");
}
fn chunk(&self) -> &[u8] {
unreachable!("chunk");
}
fn advance(&mut self, _: usize) {
unreachable!("advance");
}
fn get_u8(&mut self) -> u8 {
// specialized!
b'x'
}
}
// these should all use the specialized method
assert_eq!(Special.get_u8(), b'x');
assert_eq!((&mut Special as &mut dyn Buf).get_u8(), b'x');
assert_eq!((Box::new(Special) as Box<dyn Buf>).get_u8(), b'x');
assert_eq!(Box::new(Special).get_u8(), b'x');
}
#[test]
fn copy_to_bytes_less() {
let mut buf = &b"hello world"[..];
let bytes = buf.copy_to_bytes(5);
assert_eq!(bytes, &b"hello"[..]);
assert_eq!(buf, &b" world"[..])
}
#[test]
#[should_panic]
fn copy_to_bytes_overflow() {
let mut buf = &b"hello world"[..];
let _bytes = buf.copy_to_bytes(12);
}
+115 -26
View File
@@ -1,22 +1,23 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::{buf::IoSliceMut, BufMut, BytesMut};
use std::usize;
use std::fmt::Write;
use bytes::buf::UninitSlice;
use bytes::{BufMut, BytesMut};
use core::fmt::Write;
use core::usize;
#[test]
fn test_vec_as_mut_buf() {
let mut buf = Vec::with_capacity(64);
assert_eq!(buf.remaining_mut(), usize::MAX);
assert_eq!(buf.remaining_mut(), isize::MAX as usize);
assert!(buf.bytes_mut().len() >= 64);
assert!(buf.chunk_mut().len() >= 64);
buf.put(&b"zomg"[..]);
assert_eq!(&buf, b"zomg");
assert_eq!(buf.remaining_mut(), usize::MAX - 4);
assert_eq!(buf.remaining_mut(), isize::MAX as usize - 4);
assert_eq!(buf.capacity(), 64);
for _ in 0..16 {
@@ -26,6 +27,14 @@ fn test_vec_as_mut_buf() {
assert_eq!(buf.len(), 68);
}
#[test]
fn test_vec_put_bytes() {
let mut buf = Vec::new();
buf.push(17);
buf.put_bytes(19, 2);
assert_eq!([17, 19, 19], &buf[..]);
}
#[test]
fn test_put_u8() {
let mut buf = Vec::with_capacity(8);
@@ -45,13 +54,40 @@ fn test_put_u16() {
}
#[test]
fn test_put_int() {
let mut buf = Vec::with_capacity(8);
buf.put_int(0x1020304050607080, 3);
assert_eq!(b"\x60\x70\x80", &buf[..]);
}
#[test]
#[should_panic]
fn test_put_int_nbytes_overflow() {
let mut buf = Vec::with_capacity(8);
buf.put_int(0x1020304050607080, 9);
}
#[test]
fn test_put_int_le() {
let mut buf = Vec::with_capacity(8);
buf.put_int_le(0x1020304050607080, 3);
assert_eq!(b"\x80\x70\x60", &buf[..]);
}
#[test]
#[should_panic]
fn test_put_int_le_nbytes_overflow() {
let mut buf = Vec::with_capacity(8);
buf.put_int_le(0x1020304050607080, 9);
}
#[test]
#[should_panic(expected = "cannot advance")]
fn test_vec_advance_mut() {
// Regression test for carllerche/bytes#108.
// Verify fix for #354
let mut buf = Vec::with_capacity(8);
unsafe {
buf.advance_mut(12);
assert_eq!(buf.len(), 12);
assert!(buf.capacity() >= 12, "capacity: {}", buf.capacity());
}
}
@@ -65,25 +101,78 @@ fn test_clone() {
assert!(buf != buf2);
}
#[test]
fn test_bufs_vec_mut() {
let b1: &mut [u8] = &mut [];
let b2: &mut [u8] = &mut [];
let mut dst = [IoSliceMut::from(b1), IoSliceMut::from(b2)];
// with no capacity
let mut buf = BytesMut::new();
assert_eq!(buf.capacity(), 0);
assert_eq!(1, buf.bytes_vectored_mut(&mut dst[..]));
// with capacity
let mut buf = BytesMut::with_capacity(64);
assert_eq!(1, buf.bytes_vectored_mut(&mut dst[..]));
}
#[test]
fn test_mut_slice() {
let mut v = vec![0, 0, 0, 0];
let mut s = &mut v[..];
s.put_u32(42);
assert_eq!(s.len(), 0);
assert_eq!(&v, &[0, 0, 0, 42]);
}
#[test]
fn test_slice_put_bytes() {
let mut v = [0, 0, 0, 0];
let mut s = &mut v[..];
s.put_u8(17);
s.put_bytes(19, 2);
assert_eq!(1, s.remaining_mut());
assert_eq!(&[17, 19, 19, 0], &v[..]);
}
#[test]
fn test_deref_bufmut_forwards() {
struct Special;
unsafe impl BufMut for Special {
fn remaining_mut(&self) -> usize {
unreachable!("remaining_mut");
}
fn chunk_mut(&mut self) -> &mut UninitSlice {
unreachable!("chunk_mut");
}
unsafe fn advance_mut(&mut self, _: usize) {
unreachable!("advance");
}
fn put_u8(&mut self, _: u8) {
// specialized!
}
}
// these should all use the specialized method
Special.put_u8(b'x');
(&mut Special as &mut dyn BufMut).put_u8(b'x');
(Box::new(Special) as Box<dyn BufMut>).put_u8(b'x');
Box::new(Special).put_u8(b'x');
}
#[test]
#[should_panic]
fn write_byte_panics_if_out_of_bounds() {
let mut data = [b'b', b'a', b'r'];
let slice = unsafe { UninitSlice::from_raw_parts_mut(data.as_mut_ptr(), 3) };
slice.write_byte(4, b'f');
}
#[test]
#[should_panic]
fn copy_from_slice_panics_if_different_length_1() {
let mut data = [b'b', b'a', b'r'];
let slice = unsafe { UninitSlice::from_raw_parts_mut(data.as_mut_ptr(), 3) };
slice.copy_from_slice(b"a");
}
#[test]
#[should_panic]
fn copy_from_slice_panics_if_different_length_2() {
let mut data = [b'b', b'a', b'r'];
let slice = unsafe { UninitSlice::from_raw_parts_mut(data.as_mut_ptr(), 3) };
slice.copy_from_slice(b"abcd");
}
+184 -35
View File
@@ -1,6 +1,6 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::{Bytes, BytesMut, Buf, BufMut};
use bytes::{Buf, BufMut, Bytes, BytesMut};
use std::usize;
@@ -44,7 +44,6 @@ fn test_layout() {
mem::size_of::<Option<BytesMut>>(),
"BytesMut should be same size as Option<BytesMut>",
);
}
#[test]
@@ -87,13 +86,11 @@ fn fmt_write() {
write!(a, "{}", &s[..64]).unwrap();
assert_eq!(a, s[..64].as_bytes());
let mut b = BytesMut::with_capacity(64);
write!(b, "{}", &s[..32]).unwrap();
write!(b, "{}", &s[32..64]).unwrap();
assert_eq!(b, s[..64].as_bytes());
let mut c = BytesMut::with_capacity(64);
write!(c, "{}", s).unwrap();
assert_eq!(c, s[..].as_bytes());
@@ -176,7 +173,7 @@ fn split_off() {
#[should_panic]
fn split_off_oob() {
let mut hello = Bytes::from(&b"helloworld"[..]);
hello.split_off(44);
let _ = hello.split_off(44);
}
#[test]
@@ -273,14 +270,14 @@ fn split_to_2() {
#[should_panic]
fn split_to_oob() {
let mut hello = Bytes::from(&b"helloworld"[..]);
hello.split_to(33);
let _ = hello.split_to(33);
}
#[test]
#[should_panic]
fn split_to_oob_mut() {
let mut hello = BytesMut::from(&b"helloworld"[..]);
hello.split_to(33);
let _ = hello.split_to(33);
}
#[test]
@@ -300,16 +297,30 @@ fn split_off_to_at_gt_len() {
use std::panic;
make_bytes().split_to(4);
make_bytes().split_off(4);
let _ = make_bytes().split_to(4);
let _ = make_bytes().split_off(4);
assert!(panic::catch_unwind(move || {
make_bytes().split_to(5);
}).is_err());
let _ = make_bytes().split_to(5);
})
.is_err());
assert!(panic::catch_unwind(move || {
make_bytes().split_off(5);
}).is_err());
let _ = make_bytes().split_off(5);
})
.is_err());
}
#[test]
fn truncate() {
let s = &b"helloworld"[..];
let mut hello = Bytes::from(s);
hello.truncate(15);
assert_eq!(hello, s);
hello.truncate(10);
assert_eq!(hello, s);
hello.truncate(5);
assert_eq!(hello, "hello");
}
#[test]
@@ -330,6 +341,72 @@ fn freeze_clone_unique() {
assert_eq!(c, s);
}
#[test]
fn freeze_after_advance() {
let s = &b"abcdefgh"[..];
let mut b = BytesMut::from(s);
b.advance(1);
assert_eq!(b, s[1..]);
let b = b.freeze();
// Verify fix for #352. Previously, freeze would ignore the start offset
// for BytesMuts in Vec mode.
assert_eq!(b, s[1..]);
}
#[test]
fn freeze_after_advance_arc() {
let s = &b"abcdefgh"[..];
let mut b = BytesMut::from(s);
// Make b Arc
let _ = b.split_to(0);
b.advance(1);
assert_eq!(b, s[1..]);
let b = b.freeze();
assert_eq!(b, s[1..]);
}
#[test]
fn freeze_after_split_to() {
let s = &b"abcdefgh"[..];
let mut b = BytesMut::from(s);
let _ = b.split_to(1);
assert_eq!(b, s[1..]);
let b = b.freeze();
assert_eq!(b, s[1..]);
}
#[test]
fn freeze_after_truncate() {
let s = &b"abcdefgh"[..];
let mut b = BytesMut::from(s);
b.truncate(7);
assert_eq!(b, s[..7]);
let b = b.freeze();
assert_eq!(b, s[..7]);
}
#[test]
fn freeze_after_truncate_arc() {
let s = &b"abcdefgh"[..];
let mut b = BytesMut::from(s);
// Make b Arc
let _ = b.split_to(0);
b.truncate(7);
assert_eq!(b, s[..7]);
let b = b.freeze();
assert_eq!(b, s[..7]);
}
#[test]
fn freeze_after_split_off() {
let s = &b"abcdefgh"[..];
let mut b = BytesMut::from(s);
let _ = b.split_off(7);
assert_eq!(b, s[..7]);
let b = b.freeze();
assert_eq!(b, s[..7]);
}
#[test]
fn fns_defined_for_bytes_mut() {
let mut bytes = BytesMut::from(&b"hello world"[..]);
@@ -384,6 +461,7 @@ fn reserve_allocates_at_least_original_capacity() {
}
#[test]
#[cfg_attr(miri, ignore)] // Miri is too slow
fn reserve_max_original_capacity_value() {
const SIZE: usize = 128 * 1024;
@@ -416,7 +494,7 @@ fn reserve_vec_recycling() {
#[test]
fn reserve_in_arc_unique_does_not_overallocate() {
let mut bytes = BytesMut::with_capacity(1000);
bytes.split();
let _ = bytes.split();
// now bytes is Arc and refcount == 1
@@ -428,7 +506,7 @@ fn reserve_in_arc_unique_does_not_overallocate() {
#[test]
fn reserve_in_arc_unique_doubles() {
let mut bytes = BytesMut::with_capacity(1000);
bytes.split();
let _ = bytes.split();
// now bytes is Arc and refcount == 1
@@ -531,15 +609,15 @@ fn advance_past_len() {
#[test]
// Only run these tests on little endian systems. CI uses qemu for testing
// little endian... and qemu doesn't really support threading all that well.
#[cfg(target_endian = "little")]
// big endian... and qemu doesn't really support threading all that well.
#[cfg(any(miri, target_endian = "little"))]
fn stress() {
// Tests promoting a buffer from a vec -> shared in a concurrent situation
use std::sync::{Arc, Barrier};
use std::thread;
const THREADS: usize = 8;
const ITERS: usize = 1_000;
const ITERS: usize = if cfg!(miri) { 100 } else { 1_000 };
for i in 0..ITERS {
let data = [i as u8; 256];
@@ -706,17 +784,42 @@ fn bytes_mut_unsplit_empty_self() {
assert_eq!(b"aaabbbcccddd", &buf[..]);
}
#[test]
fn bytes_mut_unsplit_other_keeps_capacity() {
let mut buf = BytesMut::with_capacity(64);
buf.extend_from_slice(b"aabb");
// non empty other created "from" buf
let mut other = buf.split_off(buf.len());
other.extend_from_slice(b"ccddee");
buf.unsplit(other);
assert_eq!(buf.capacity(), 64);
}
#[test]
fn bytes_mut_unsplit_empty_other_keeps_capacity() {
let mut buf = BytesMut::with_capacity(64);
buf.extend_from_slice(b"aabbccddee");
// empty other created "from" buf
let other = buf.split_off(buf.len());
buf.unsplit(other);
assert_eq!(buf.capacity(), 64);
}
#[test]
fn bytes_mut_unsplit_arc_different() {
let mut buf = BytesMut::with_capacity(64);
buf.extend_from_slice(b"aaaabbbbeeee");
buf.split_off(8); //arc
let _ = buf.split_off(8); //arc
let mut buf2 = BytesMut::with_capacity(64);
buf2.extend_from_slice(b"ccccddddeeee");
buf2.split_off(8); //arc
let _ = buf2.split_off(8); //arc
buf.unsplit(buf2);
assert_eq!(b"aaaabbbbccccdddd", &buf[..]);
@@ -786,7 +889,6 @@ fn slice_ref_works() {
test_slice_ref(&bytes, 9, 9, b"");
}
#[test]
fn slice_ref_empty() {
let bytes = Bytes::from(&b""[..]);
@@ -796,6 +898,16 @@ fn slice_ref_empty() {
assert_eq!(&sub[..], b"");
}
#[test]
fn slice_ref_empty_subslice() {
let bytes = Bytes::from(&b"abcde"[..]);
let subbytes = bytes.slice(0..0);
let slice = &subbytes[..];
// The `slice` object is derived from the original `bytes` object
// so `slice_ref` should work.
assert_eq!(Bytes::new(), bytes.slice_ref(slice));
}
#[test]
#[should_panic]
fn slice_ref_catches_not_a_subset() {
@@ -806,21 +918,19 @@ fn slice_ref_catches_not_a_subset() {
}
#[test]
#[should_panic]
fn slice_ref_catches_not_an_empty_subset() {
fn slice_ref_not_an_empty_subset() {
let bytes = Bytes::from(&b"012345678"[..]);
let slice = &b""[0..0];
bytes.slice_ref(slice);
assert_eq!(Bytes::new(), bytes.slice_ref(slice));
}
#[test]
#[should_panic]
fn empty_slice_ref_catches_not_an_empty_subset() {
let bytes = Bytes::copy_from_slice(&b""[..]);
let slice = &b""[0..0];
fn empty_slice_ref_not_an_empty_subset() {
let bytes = Bytes::new();
let slice = &b"some other slice"[0..0];
bytes.slice_ref(slice);
assert_eq!(Bytes::new(), bytes.slice_ref(slice));
}
#[test]
@@ -828,23 +938,39 @@ fn bytes_buf_mut_advance() {
let mut bytes = BytesMut::with_capacity(1024);
unsafe {
let ptr = bytes.bytes_mut().as_ptr();
assert_eq!(1024, bytes.bytes_mut().len());
let ptr = bytes.chunk_mut().as_mut_ptr();
assert_eq!(1024, bytes.chunk_mut().len());
bytes.advance_mut(10);
let next = bytes.bytes_mut().as_ptr();
assert_eq!(1024 - 10, bytes.bytes_mut().len());
let next = bytes.chunk_mut().as_mut_ptr();
assert_eq!(1024 - 10, bytes.chunk_mut().len());
assert_eq!(ptr.offset(10), next);
// advance to the end
bytes.advance_mut(1024 - 10);
// The buffer size is doubled
assert_eq!(1024, bytes.bytes_mut().len());
assert_eq!(1024, bytes.chunk_mut().len());
}
}
#[test]
fn bytes_buf_mut_reuse_when_fully_consumed() {
use bytes::{Buf, BytesMut};
let mut buf = BytesMut::new();
buf.reserve(8192);
buf.extend_from_slice(&[0u8; 100][..]);
let p = &buf[0] as *const u8;
buf.advance(100);
buf.reserve(8192);
buf.extend_from_slice(b" ");
assert_eq!(&buf[0] as *const u8, p);
}
#[test]
#[should_panic]
fn bytes_reserve_overflow() {
@@ -853,3 +979,26 @@ fn bytes_reserve_overflow() {
bytes.reserve(usize::MAX);
}
#[test]
fn bytes_with_capacity_but_empty() {
// See https://github.com/tokio-rs/bytes/issues/340
let vec = Vec::with_capacity(1);
let _ = Bytes::from(vec);
}
#[test]
fn bytes_put_bytes() {
let mut bytes = BytesMut::new();
bytes.put_u8(17);
bytes.put_bytes(19, 2);
assert_eq!([17, 19, 19], bytes.as_ref());
}
#[test]
fn box_slice_empty() {
// See https://github.com/tokio-rs/bytes/issues/340
let empty: Box<[u8]> = Default::default();
let b = Bytes::from(empty);
assert!(b.is_empty());
}
+69
View File
@@ -0,0 +1,69 @@
//! Test using `Bytes` with an allocator that hands out "odd" pointers for
//! vectors (pointers where the LSB is set).
#![cfg(not(miri))] // Miri does not support custom allocators (also, Miri is "odd" by default with 50% chance)
use std::alloc::{GlobalAlloc, Layout, System};
use std::ptr;
use bytes::Bytes;
#[global_allocator]
static ODD: Odd = Odd;
struct Odd;
unsafe impl GlobalAlloc for Odd {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
if layout.align() == 1 && layout.size() > 0 {
// Allocate slightly bigger so that we can offset the pointer by 1
let size = layout.size() + 1;
let new_layout = match Layout::from_size_align(size, 1) {
Ok(layout) => layout,
Err(_err) => return ptr::null_mut(),
};
let ptr = System.alloc(new_layout);
if !ptr.is_null() {
let ptr = ptr.offset(1);
ptr
} else {
ptr
}
} else {
System.alloc(layout)
}
}
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
if layout.align() == 1 && layout.size() > 0 {
let size = layout.size() + 1;
let new_layout = match Layout::from_size_align(size, 1) {
Ok(layout) => layout,
Err(_err) => std::process::abort(),
};
System.dealloc(ptr.offset(-1), new_layout);
} else {
System.dealloc(ptr, layout);
}
}
}
#[test]
fn sanity_check_odd_allocator() {
let vec = vec![33u8; 1024];
let p = vec.as_ptr() as usize;
assert!(p & 0x1 == 0x1, "{:#b}", p);
}
#[test]
fn test_bytes_from_vec_drop() {
let vec = vec![33u8; 1024];
let _b = Bytes::from(vec);
}
#[test]
fn test_bytes_clone_drop() {
let vec = vec![33u8; 1024];
let b1 = Bytes::from(vec);
let _b2 = b1.clone();
}
+79
View File
@@ -0,0 +1,79 @@
use std::alloc::{GlobalAlloc, Layout, System};
use std::{mem, ptr};
use bytes::{Buf, Bytes};
#[global_allocator]
static LEDGER: Ledger = Ledger;
struct Ledger;
const USIZE_SIZE: usize = mem::size_of::<usize>();
unsafe impl GlobalAlloc for Ledger {
unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
if layout.align() == 1 && layout.size() > 0 {
// Allocate extra space to stash a record of
// how much space there was.
let orig_size = layout.size();
let size = orig_size + USIZE_SIZE;
let new_layout = match Layout::from_size_align(size, 1) {
Ok(layout) => layout,
Err(_err) => return ptr::null_mut(),
};
let ptr = System.alloc(new_layout);
if !ptr.is_null() {
(ptr as *mut usize).write(orig_size);
let ptr = ptr.offset(USIZE_SIZE as isize);
ptr
} else {
ptr
}
} else {
System.alloc(layout)
}
}
unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
if layout.align() == 1 && layout.size() > 0 {
let off_ptr = (ptr as *mut usize).offset(-1);
let orig_size = off_ptr.read();
if orig_size != layout.size() {
panic!(
"bad dealloc: alloc size was {}, dealloc size is {}",
orig_size,
layout.size()
);
}
let new_layout = match Layout::from_size_align(layout.size() + USIZE_SIZE, 1) {
Ok(layout) => layout,
Err(_err) => std::process::abort(),
};
System.dealloc(off_ptr as *mut u8, new_layout);
} else {
System.dealloc(ptr, layout);
}
}
}
#[test]
fn test_bytes_advance() {
let mut bytes = Bytes::from(vec![10, 20, 30]);
bytes.advance(1);
drop(bytes);
}
#[test]
fn test_bytes_truncate() {
let mut bytes = Bytes::from(vec![10, 20, 30]);
bytes.truncate(2);
drop(bytes);
}
#[test]
fn test_bytes_truncate_and_advance() {
let mut bytes = Bytes::from(vec![10, 20, 30]);
bytes.truncate(2);
bytes.advance(1);
drop(bytes);
}
+29 -7
View File
@@ -1,7 +1,7 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::{Buf, BufMut, Bytes};
use bytes::buf::{BufExt, BufMutExt};
#[cfg(feature = "std")]
use std::io::IoSlice;
#[test]
@@ -9,7 +9,7 @@ fn collect_two_bufs() {
let a = Bytes::from(&b"hello"[..]);
let b = Bytes::from(&b"world"[..]);
let res = a.chain(b).to_bytes();
let res = a.chain(b).copy_to_bytes(10);
assert_eq!(res, &b"helloworld"[..]);
}
@@ -42,6 +42,7 @@ fn iterating_two_bufs() {
assert_eq!(res, &b"helloworld"[..]);
}
#[cfg(feature = "std")]
#[test]
fn vectored_read() {
let a = Bytes::from(&b"hello"[..]);
@@ -61,7 +62,7 @@ fn vectored_read() {
IoSlice::new(b4),
];
assert_eq!(2, buf.bytes_vectored(&mut iovecs));
assert_eq!(2, buf.chunks_vectored(&mut iovecs));
assert_eq!(iovecs[0][..], b"hello"[..]);
assert_eq!(iovecs[1][..], b"world"[..]);
assert_eq!(iovecs[2][..], b""[..]);
@@ -82,7 +83,7 @@ fn vectored_read() {
IoSlice::new(b4),
];
assert_eq!(2, buf.bytes_vectored(&mut iovecs));
assert_eq!(2, buf.chunks_vectored(&mut iovecs));
assert_eq!(iovecs[0][..], b"llo"[..]);
assert_eq!(iovecs[1][..], b"world"[..]);
assert_eq!(iovecs[2][..], b""[..]);
@@ -103,7 +104,7 @@ fn vectored_read() {
IoSlice::new(b4),
];
assert_eq!(1, buf.bytes_vectored(&mut iovecs));
assert_eq!(1, buf.chunks_vectored(&mut iovecs));
assert_eq!(iovecs[0][..], b"world"[..]);
assert_eq!(iovecs[1][..], b""[..]);
assert_eq!(iovecs[2][..], b""[..]);
@@ -124,10 +125,31 @@ fn vectored_read() {
IoSlice::new(b4),
];
assert_eq!(1, buf.bytes_vectored(&mut iovecs));
assert_eq!(1, buf.chunks_vectored(&mut iovecs));
assert_eq!(iovecs[0][..], b"ld"[..]);
assert_eq!(iovecs[1][..], b""[..]);
assert_eq!(iovecs[2][..], b""[..]);
assert_eq!(iovecs[3][..], b""[..]);
}
}
#[test]
fn chain_get_bytes() {
let mut ab = Bytes::copy_from_slice(b"ab");
let mut cd = Bytes::copy_from_slice(b"cd");
let ab_ptr = ab.as_ptr();
let cd_ptr = cd.as_ptr();
let mut chain = (&mut ab).chain(&mut cd);
let a = chain.copy_to_bytes(1);
let bc = chain.copy_to_bytes(2);
let d = chain.copy_to_bytes(1);
assert_eq!(Bytes::copy_from_slice(b"a"), a);
assert_eq!(Bytes::copy_from_slice(b"bc"), bc);
assert_eq!(Bytes::copy_from_slice(b"d"), d);
// assert `get_bytes` did not allocate
assert_eq!(ab_ptr, a.as_ptr());
// assert `get_bytes` did not allocate
assert_eq!(cd_ptr.wrapping_offset(1), d.as_ptr());
}
+1 -1
View File
@@ -1,4 +1,4 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::Bytes;
+1 -2
View File
@@ -1,4 +1,4 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::Bytes;
@@ -11,7 +11,6 @@ fn iter_len() {
assert_eq!(iter.len(), 11);
}
#[test]
fn empty_iter_len() {
let buf = Bytes::from_static(b"");
+5 -4
View File
@@ -1,14 +1,15 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
#![cfg(feature = "std")]
use std::io::{BufRead, Read};
use bytes::buf::{BufExt};
use bytes::Buf;
#[test]
fn read() {
let buf1 = &b"hello "[..];
let buf2 = &b"world"[..];
let buf = BufExt::chain(buf1, buf2); // Disambiguate with Read::chain
let buf = Buf::chain(buf1, buf2); // Disambiguate with Read::chain
let mut buffer = Vec::new();
buf.reader().read_to_end(&mut buffer).unwrap();
assert_eq!(b"hello world", &buffer[..]);
@@ -18,7 +19,7 @@ fn read() {
fn buf_read() {
let buf1 = &b"hell"[..];
let buf2 = &b"o\nworld"[..];
let mut reader = BufExt::chain(buf1, buf2).reader();
let mut reader = Buf::chain(buf1, buf2).reader();
let mut line = String::new();
reader.read_line(&mut line).unwrap();
assert_eq!("hello\n", &line);
+2 -2
View File
@@ -1,7 +1,7 @@
#![cfg(feature = "serde")]
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use serde_test::{Token, assert_tokens};
use serde_test::{assert_tokens, Token};
#[test]
fn test_ser_de_empty() {
+23 -3
View File
@@ -1,6 +1,7 @@
#![deny(warnings, rust_2018_idioms)]
#![warn(rust_2018_idioms)]
use bytes::buf::{Buf, BufExt};
use bytes::buf::Buf;
use bytes::Bytes;
#[test]
fn long_take() {
@@ -8,5 +9,24 @@ fn long_take() {
// overrun the buffer. Regression test for #138.
let buf = b"hello world".take(100);
assert_eq!(11, buf.remaining());
assert_eq!(b"hello world", buf.bytes());
assert_eq!(b"hello world", buf.chunk());
}
#[test]
fn take_copy_to_bytes() {
let mut abcd = Bytes::copy_from_slice(b"abcd");
let abcd_ptr = abcd.as_ptr();
let mut take = (&mut abcd).take(2);
let a = take.copy_to_bytes(1);
assert_eq!(Bytes::copy_from_slice(b"a"), a);
// assert `to_bytes` did not allocate
assert_eq!(abcd_ptr, a.as_ptr());
assert_eq!(Bytes::copy_from_slice(b"bcd"), abcd);
}
#[test]
#[should_panic]
fn take_copy_to_bytes_panics() {
let abcd = Bytes::copy_from_slice(b"abcd");
abcd.take(2).copy_to_bytes(3);
}