Flutter Dart Dio Get Request is so slow

Question

I am hosting a space in digital ocean - it is basically Amazon S3 equalivant of digital ocean. My problem with dio is, I am making a get request with dio to a file of 10MB size. The request takes around 9 seconds on my phone but 3 seconds in my browser. I also had this issue in my custom backend. Get requests made with dio (which uses http module of dart) seems to be extremely slow. I need to solve this issue as I need to transfer 50MB of data to user from time to time. Why is dio acting slow on GET requests?

I suspect this might be the underlying cause check here

await Dio().get(
        "Remote_Url_I_can_not_share",
        onReceiveProgress: (int downloaded, int total) {
          listener
              .call((downloaded.toDouble() / total.toDouble() * metadataPerc));
        },
        cancelToken: _cancelToken,
      ).catchError((err) => throw err);

Answer 1

I believe that reason for this; buffer size is limited to 8KB somewhere underlying.

I tried whole day to increase it. Still no success. Let me share my experience with that buffer size.

Imagine you're downloading a file which is 16 mb.

Please consider that remote server has also higher speed than your download speed. (I mean just forget about server load, etc.. )

If buffersize:

128 bytes, downloading 16mb file takes : 10.820 seconds

1024 bytes, downloading 16mb file takes : 6.276 seconds

8192 bytes, downloading 16mb file takes : 4.776 seconds

16384 bytes, downloading 16mb file takes : 3.759 seconds

32768 bytes, downloading 16mb file takes : 2.956 seconds

------- After This, If we increase chunk size, download time is also increasing

65536 bytes, downloading 16mb file takes : 4.186 seconds

131072 bytes, downloading 16mb file takes : 5.250 seconds

524288 bytes, downloading 16mb file takes : 7.460 seconds

So somehow, if you can set that buffersize 16k or 32k rather than 8k, I believe download speed will increase.

Please feel free to test your results (I got 3 tries and got average of them for the timings)

package dltest;

import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;

public class DLTest 
{
    public static void main(String[] args) throws Exception 
    {

        String filePath = "http://hcmaslov.d-real.sci-nnov.ru/public/mp3/Metallica/Metallica%20'...And%20Justice%20For%20All'.mp3";

        URL url = new URL(filePath);
        URLConnection uc = url.openConnection();
        InputStream is = uc.getInputStream();

        long start = System.currentTimeMillis();

        int downloaded = 0;
        int total = uc.getContentLength();
        int partialRead = 0;

        // byte chunk[] = new byte[128];
        // byte chunk[] = new byte[1024];
        // byte chunk[] = new byte[4096];
        // byte chunk[] = new byte[8192];
           byte chunk[] = new byte[16384];
        // byte chunk[] = new byte[32768];
        // byte chunk[] = new byte[524288];

        while ( (partialRead = is.read(chunk)) != -1)
        {
                // Print If You Like..
        }

        is.close();
        long end = System.currentTimeMillis();

        System.out.println("Chunk Size ["+(chunk.length)+"] Time To Complete :  "+(end - start));
    }
}

Answer 2

My experience with DigitalOcean Spaces has been a very fluctuating one. DO Spaces is, in my opinion, not production ready. I was using their CDN feature for a website, and sometimes the response times would be about 20ms, but sometimes they would exceed 6 seconds. This was in AMS3 datacenter region.

Can you confirm this happens with other S3/servers as well? Such as gstatic, or Amazon CloudFront CDN?

This fluctuating behaviour happened constantly, which is why we transferred all our assets to Amazon S3 + CloudFront. It provides much more consistent results.

It could be that the phone you are testing on uses a very unoptimized traceroute to the DigitalOcean datacenters. That's why you should try different servers.