Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Downloading large number of files using Expo.Filesystem.downloadAsync sometimes stuck indefinitely

Im using Expo.Filesystem.downloadAsync to download large no. of files like images & videos. But it sometimes stuck indefinitely at some point. Im tying to download files inside a loop. The code is :

        let image_downloading = section.map(async (item, i) => {
            item.image !== null  ?
                await FileSystem.downloadAsync(item.image,
                    directory + item.image.split("/").reverse()[0]
                )
                    .then(({ uri }) => {
                        item['image'] = uri;
                        console.log('Finished downloading section to ', uri);
                    })
                    .catch(({error}) => {
                        console.log('errorrrrrrrrrrrrr',error)
                    })
                : null
    });
    await Promise.all(image_downloading);

Also i've tried using FileSystem.createDownloadResumable. When using createDownloadResumable the downloading is getting very slow

like image 771
sachin mathew Avatar asked Jun 20 '19 05:06

sachin mathew


1 Answers

The actual problem was with the server i was sending request to download file.

It freezes on getting a large number of requests at a time.

so i changed my function to send only 20 requests at a time and wait for a second before sending the next 20.

For that first i split my array into chunks of the same size

let item_chunk_size = 20;
let itemArray = [];
for (let i = 0;i<items.length; i+= item_chunk_size)  {
    let myChunk = items.slice(i, i+item_chunk_size);
    itemArray.push(myChunk)
}

then downloaded images by sending 20 requests at a time

for (let i=0;i<itemArray.length;i++){
    let itemChunk = itemArray[i].map(async item => {
        if(item.image !== '' && item.image){
            await FileSystem.downloadAsync(
                item.image,
                directory + item.image.split("/").reverse()[0]
            )
                .then(({uri}) => {
                    this.setState({count:this.state.count+1});
                    item['image'] = uri;
                    console.log('Finished downloading section to ', uri);
                })
        }
        if(item.video !== '' && item.video){
            await FileSystem.downloadAsync(
                item.video,
                directory + item.video.split("/").reverse()[0]
            )
                .then(({uri}) => {
                    this.setState({count:this.state.count+1});
                    item['video'] = uri;
                    console.log('Finished downloading section to ', uri);
                })
        }
    });
    await Promise.all(itemChunk);
    await this.wait(1000);
}

The function to wait for a second after 20 requests

 wait = async(ms) => {
    return new Promise(resolve => {
        setTimeout(resolve, ms);
    })
}
like image 146
sachin mathew Avatar answered Oct 23 '22 05:10

sachin mathew