fs: add support for non-threadpool executors (#1495)

Provides a thread pool dedicated to running blocking operations (#588)
and update `tokio-fs` to use this pool.

In an effort to make incremental progress, this is an initial step
towards a final solution. First, it provides a very basic pool
implementation with the intend that the pool will be
replaced before the final release. Second, it updates `tokio-fs` to
always use this blocking pool instead of conditionally using
`threadpool::blocking`. Issue #588 contains additional discussion around
potential improvements to the "blocking for all" strategy.

The implementation provided here builds on work started in #954 and
continued in #1045. The general idea is th same as #1045, but the PR
improves on some of the details:

* The number of explicit operations tracked by `File` is reduced only to
  the ones that could interact. All other ops are spawned on the
  blocking pool without being tracked by the `File` instance.

* The `seek` implementation is not backed by a trait and `poll_seek`
  function. This avoids the question of how to model non-blocking seeks
  on top of a blocking file. In this patch, `seek` is represented as an
  `async fn`. If the associated future is dropped before the caller
  observes the return value, we make no effort to define the state in
  which the file ends up.
This commit is contained in:
Carl Lerche
2019-08-27 12:25:20 -07:00
committed by GitHub
parent 08099bb2d3
commit 08e20fcf6a
35 changed files with 1974 additions and 263 deletions
+43
View File
@@ -1,12 +1,54 @@
#![warn(rust_2018_idioms)]
use tokio::fs::File;
use tokio::prelude::*;
use std::io::prelude::*;
use tempfile::NamedTempFile;
/*
use rand::{distributions, thread_rng, Rng};
use std::fs;
use std::io::SeekFrom;
use tempfile::Builder as TmpBuilder;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio_fs::*;
*/
const HELLO: &[u8] = b"hello world...";
#[tokio::test]
async fn basic_read() {
let mut tempfile = tempfile();
tempfile.write_all(HELLO).unwrap();
let mut file = File::open(tempfile.path()).await.unwrap();
let mut buf = [0; 1024];
let n = file.read(&mut buf).await.unwrap();
assert_eq!(n, HELLO.len());
assert_eq!(&buf[..n], HELLO);
}
#[tokio::test]
async fn basic_write() {
let tempfile = tempfile();
let mut file = File::create(tempfile.path()).await.unwrap();
file.write_all(HELLO).await.unwrap();
file.flush().await.unwrap();
let file = std::fs::read(tempfile.path()).unwrap();
assert_eq!(file, HELLO);
}
fn tempfile() -> NamedTempFile {
NamedTempFile::new().unwrap()
}
/*
mod pool;
#[test]
@@ -154,3 +196,4 @@ fn clone() {
assert_eq!(dst, b"clone successful")
}
*/