How to write a hyper response body to a file?

Question

I am trying to write a test program with tokio that grabs a file from a website and writes the streamed response to a file. The hyper website shows an example that uses a while loop and uses the .data() method the response body, but I'd like to manipulate the stream with .map() and a couple others.

I thought the next reasonable thing to try would be to convert the stream to an AsyncRead by using the .into_async_read() method from TryStreamExt, but that doesn't seem to work. I had to use a map to convert the hyper::error::Error into a std::error::Error to get a TryStream, but now the compiler is telling me that AsyncRead isn't implemented for the transformed stream. Here is my main.rs file and the error:

src/main.rs

use futures::stream::{StreamExt, TryStreamExt};
use http::Request;
use hyper::{Body, Client};
use hyper_tls::HttpsConnector;
use tokio::fs::File;
use tokio::io;

use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let https = HttpsConnector::new();
    let client = Client::builder().build::<_, Body>(https);

    let request = Request::get("some file from the internet").body(Body::empty())?;
    let response = client.request(request).await?;

    let mut stream = response
        .body()
        .map(|result| result.map_err(|error| std::io::Error::new(std::io::ErrorKind::Other, "Error!")))
        .into_async_read();
    let mut file = File::create("output file").await?;

    io::copy(&mut stream, &mut file).await?;

    Ok(())
}

Error

error[E0277]: the trait bound `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure@src/main.rs:20:14: 20:103]>>: tokio::io::async_read::AsyncRead` is not satisfied
  --> src/main.rs:24:5
   |
24 |     io::copy(&mut stream, &mut file).await?;
   |     ^^^^^^^^ the trait `tokio::io::async_read::AsyncRead` is not implemented for `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure@src/main.rs:20:14: 20:103]>>`
   | 
  ::: /Users/jackson/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.13/src/io/util/copy.rs:63:12
   |
63 |         R: AsyncRead + Unpin + ?Sized,
   |            --------- required by this bound in `tokio::io::util::copy::copy`

error[E0277]: the trait bound `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure@src/main.rs:20:14: 20:103]>>: tokio::io::async_read::AsyncRead` is not satisfied
  --> src/main.rs:24:5
   |
24 |     io::copy(&mut stream, &mut file).await?;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `tokio::io::async_read::AsyncRead` is not implemented for `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure@src/main.rs:20:14: 20:103]>>`
   |
   = note: required because of the requirements on the impl of `core::future::future::Future` for `tokio::io::util::copy::Copy<'_, futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure@src/main.rs:20:14: 20:103]>>, tokio::fs::file::File>`

Answer 1

You almost had it. You invoked into_async_read which gives you an implementation of futures::io::AsyncRead but you want a tokio::io::AsyncRead.

The tokio-util crate gives you a tool to do this conversion.

Add to your Cargo.toml:

tokio-util = { version = "0.3.1", features=["compat"] }

And say you add a conversion function like this:

fn to_tokio_async_read(r: impl futures::io::AsyncRead) -> impl tokio::io::AsyncRead {
    tokio_util::compat::FuturesAsyncReadCompatExt::compat(r)
}

then your code could become:

let mut futures_io_async_read = response
        .body()
        .map(|result| result.map_err(|error| std::io::Error::new(std::io::ErrorKind::Other, "Error!")))
        .into_async_read();

let tokio_async_read = to_tokio_async_read(futures_io_async_read)

let mut file = File::create("output file").await?;

io::copy(&mut tokio_async_read, &mut file).await?;