Progress bar with HttpClient

Question

i have a file downloader function:

        HttpClientHandler aHandler = new HttpClientHandler();
        aHandler.ClientCertificateOptions = ClientCertificateOption.Automatic;
        HttpClient aClient = new HttpClient(aHandler);
        aClient.DefaultRequestHeaders.ExpectContinue = false;
        HttpResponseMessage response = await aClient.GetAsync(url);
        InMemoryRandomAccessStream randomAccessStream = new InMemoryRandomAccessStream();

        // To save downloaded image to local storage
        var imageFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(
        filename, CreationCollisionOption.ReplaceExisting);
        var fs = await imageFile.OpenAsync(FileAccessMode.ReadWrite);
        DataWriter writer = new DataWriter(fs.GetOutputStreamAt(0));

        writer.WriteBytes(await response.Content.ReadAsByteArrayAsync());

        await writer.StoreAsync();
        //current.image.SetSource(randomAccessStream);
        writer.DetachStream();
        await fs.FlushAsync();

How can i realize progress bar functionality? Maybe i can get the writers bytes written so far? Or something?

P.S. I cant use DownloadOperation(Background transferring) because data from server requests certificate - and this functionality doesn't exist in DownloadOperations.

Answer 1

From .Net 4.5 onwards: Use IProgress<T>

Since .Net 4.5 you can handle asynchronous progress reporting with the IProgress<T> interface. You can write an extension method for downloading files using the HttpClient that can be called like this where progress is the implementation of IProgress<float> for your progress bar or other UI stuff:

// Seting up the http client used to download the data
using (var client = new HttpClient()) {
    client.Timeout = TimeSpan.FromMinutes(5);

    // Create a file stream to store the downloaded data.
    // This really can be any type of writeable stream.
    using (var file = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.None)) {

        // Use the custom extension method below to download the data.
        // The passed progress-instance will receive the download status updates.
        await client.DownloadAsync(DownloadUrl, file, progress, cancellationToken);
    }
}

Implementation

The code for this extension method looks like this. Note that this extension depends on another extension for handling asynchronous stream copying with progress reporting.

public static class HttpClientExtensions
{
    public static async Task DownloadAsync(this HttpClient client, string requestUri, Stream destination, IProgress<float> progress = null, CancellationToken cancellationToken = default) {
        // Get the http headers first to examine the content length
        using (var response = await client.GetAsync(requestUri, HttpCompletionOption.ResponseHeadersRead)) {
            var contentLength = response.Content.Headers.ContentLength;

            using (var download = await response.Content.ReadAsStreamAsync()) {

                // Ignore progress reporting when no progress reporter was 
                // passed or when the content length is unknown
                if (progress == null || !contentLength.HasValue) {
                    await download.CopyToAsync(destination);
                    return;
                }

                // Convert absolute progress (bytes downloaded) into relative progress (0% - 100%)
                var relativeProgress = new Progress<long>(totalBytes => progress.Report((float)totalBytes / contentLength.Value));
                // Use extension method to report progress while downloading
                await download.CopyToAsync(destination, 81920, relativeProgress, cancellationToken);
                progress.Report(1);
            }
        }
    }
}

With stream extension for the real progress reporting:

public static class StreamExtensions
{
    public static async Task CopyToAsync(this Stream source, Stream destination, int bufferSize, IProgress<long> progress = null, CancellationToken cancellationToken = default) {
        if (source == null)
            throw new ArgumentNullException(nameof(source));
        if (!source.CanRead)
            throw new ArgumentException("Has to be readable", nameof(source));
        if (destination == null)
            throw new ArgumentNullException(nameof(destination));
        if (!destination.CanWrite)
            throw new ArgumentException("Has to be writable", nameof(destination));
        if (bufferSize < 0)
            throw new ArgumentOutOfRangeException(nameof(bufferSize));

        var buffer = new byte[bufferSize];
        long totalBytesRead = 0;
        int bytesRead;
        while ((bytesRead = await source.ReadAsync(buffer, 0, buffer.Length, cancellationToken).ConfigureAwait(false)) != 0) {
            await destination.WriteAsync(buffer, 0, bytesRead, cancellationToken).ConfigureAwait(false);
            totalBytesRead += bytesRead;
            progress?.Report(totalBytesRead);
        }
    }
}

Answer 2

Here's a self-contained class that'll do the download, and report back the progress percentage, based on code from TheBlueSky on this SO answer, and eriksendc on this GitHub comment.

public class HttpClientDownloadWithProgress : IDisposable
{
    private readonly string _downloadUrl;
    private readonly string _destinationFilePath;

    private HttpClient _httpClient;

    public delegate void ProgressChangedHandler(long? totalFileSize, long totalBytesDownloaded, double? progressPercentage);

    public event ProgressChangedHandler ProgressChanged;

    public HttpClientDownloadWithProgress(string downloadUrl, string destinationFilePath)
    {
        _downloadUrl = downloadUrl;
        _destinationFilePath = destinationFilePath;
    }

    public async Task StartDownload()
    {
        _httpClient = new HttpClient { Timeout = TimeSpan.FromDays(1) };

        using (var response = await _httpClient.GetAsync(_downloadUrl, HttpCompletionOption.ResponseHeadersRead))
            await DownloadFileFromHttpResponseMessage(response);
    }

    private async Task DownloadFileFromHttpResponseMessage(HttpResponseMessage response)
    {
        response.EnsureSuccessStatusCode();

        var totalBytes = response.Content.Headers.ContentLength;

        using (var contentStream = await response.Content.ReadAsStreamAsync())
            await ProcessContentStream(totalBytes, contentStream);
    }

    private async Task ProcessContentStream(long? totalDownloadSize, Stream contentStream)
    {
        var totalBytesRead = 0L;
        var readCount = 0L;
        var buffer = new byte[8192];
        var isMoreToRead = true;

        using (var fileStream = new FileStream(_destinationFilePath, FileMode.Create, FileAccess.Write, FileShare.None, 8192, true))
        {
            do
            {
                var bytesRead = await contentStream.ReadAsync(buffer, 0, buffer.Length);
                if (bytesRead == 0)
                {
                    isMoreToRead = false;
                    TriggerProgressChanged(totalDownloadSize, totalBytesRead);
                    continue;
                }

                await fileStream.WriteAsync(buffer, 0, bytesRead);

                totalBytesRead += bytesRead;
                readCount += 1;

                if (readCount % 100 == 0)
                    TriggerProgressChanged(totalDownloadSize, totalBytesRead);
            }
            while (isMoreToRead);
        }
    }

    private void TriggerProgressChanged(long? totalDownloadSize, long totalBytesRead)
    {
        if (ProgressChanged == null)
            return;

        double? progressPercentage = null;
        if (totalDownloadSize.HasValue)
            progressPercentage = Math.Round((double)totalBytesRead / totalDownloadSize.Value * 100, 2);

        ProgressChanged(totalDownloadSize, totalBytesRead, progressPercentage);
    }

    public void Dispose()
    {
        _httpClient?.Dispose();
    }
}

Usage:

var downloadFileUrl = "http://example.com/file.zip";
var destinationFilePath = Path.GetFullPath("file.zip");

using (var client = new HttpClientDownloadWithProgress(downloadFileUrl, destinationFilePath))
{
    client.ProgressChanged += (totalFileSize, totalBytesDownloaded, progressPercentage) => {
        Console.WriteLine($"{progressPercentage}% ({totalBytesDownloaded}/{totalFileSize})");
    };

    await client.StartDownload();
}

Result:

7.81% (26722304/342028776)
8.05% (27535016/342028776)
8.28% (28307984/342028776)
8.5% (29086548/342028776)
8.74% (29898692/342028776)
8.98% (30704184/342028776)
9.22% (31522816/342028776)

Answer 3

The best way to go is using Windows.Web.Http.HttpClient instead of System.Net.Http.HttpClient. The first one supports progress.

But if for some reason you want to stick to the System.Net one, you will need to implement your own progress.

Remove the DataWriter, remove the InMemoryRandomAccessStream and add HttpCompletionOption.ResponseHeadersRead to GetAsync call so it returns as soon as headers are received, not when the whole response is received. I.e.:

// Your original code.
HttpClientHandler aHandler = new HttpClientHandler();
aHandler.ClientCertificateOptions = ClientCertificateOption.Automatic;
HttpClient aClient = new HttpClient(aHandler);
aClient.DefaultRequestHeaders.ExpectContinue = false;
HttpResponseMessage response = await aClient.GetAsync(
    url,
    HttpCompletionOption.ResponseHeadersRead); // Important! ResponseHeadersRead.

// To save downloaded image to local storage
var imageFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(
    filename,
    CreationCollisionOption.ReplaceExisting);
var fs = await imageFile.OpenAsync(FileAccessMode.ReadWrite);

// New code.
Stream stream = await response.Content.ReadAsStreamAsync();
IInputStream inputStream = stream.AsInputStream();
ulong totalBytesRead = 0;
while (true)
{
    // Read from the web.
    IBuffer buffer = new Windows.Storage.Streams.Buffer(1024);
    buffer = await inputStream.ReadAsync(
        buffer,
        buffer.Capacity,
        InputStreamOptions.None);

    if (buffer.Length == 0)
    {
        // There is nothing else to read.
        break;
    }

    // Report progress.
    totalBytesRead += buffer.Length;
    System.Diagnostics.Debug.WriteLine("Bytes read: {0}", totalBytesRead);

    // Write to file.
    await fs.WriteAsync(buffer);
}
inputStream.Dispose();
fs.Dispose();