Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Transfer large files between Android and Node.js

I have a photo-taking Android app (written in React Native, but I can add native modules if need be) that pairs with a gallery-type app on the user's computer (written in Electron with React). The number of photos can be large (we've had upwards of a thousand photos at a time), and they need to be transferred, uncompressed and uncropped, to the computer app. I need a reliable way to do this without routing the data through the Internet first.

Things I've tried

Node USB: Looks like a good library, but is unusable without installing a driver on the user's computer. The driver then appears to replace the default Windows USB driver, and the phone becomes inaccessible through normal means (opening a file explorer in Windows). Perhaps there's some way to switch drivers "on the fly", using the driver only while transferring photos, and then switching it back out for the system driver once we're done?

ADB: This is the original solution I tried, and it does exactly what I need it to, with two dealbreaking problems: for one, the phone needs to have Dev Mode enabled, and for two, ADB doesn't seem to recognize all devices (we've had a lot of trouble getting it to work with an LG V20, for example). The ideal answer to this question would be something like ADB without those two issues.

WiFi routing: we connect the phone to the same WiFi network as the host computer, use WebSockets to handshake IPs through the Internet, and then initiate a direct connection over the WiFi network. If I cannot find any other solutions, this will likely be the one I'll go for. The biggest problem with this option is the relatively large amount of manual setup required. Our users don't tend to be very technically savvy, and I don't know how reasonable it is to assume they will have WiFi in their home. At any rate, this cannot be the only option for transfer. Further, transferring gigabyte(s) of photos over the wireless network will cause a slowdown for everyone on the same network.

Tether routing: we tether the phone to the computer, and initiate a direct connection much in the same way as in the above step. The downsides to this method are that only phones with SIM cards can activate tethering (not all of our phones have SIM cards, as connectivity isn't their primary purpose for our needs), plus all of the other traffic on the computer will now be routed through the phone, which would be bad for customers' data caps who do have SIM cards. Additionally, there is no way to programmatically turn tethering on, there is only the ability to bring up the tethering menu; it's not too big a dealbreaker, but the other two very much are.

Bluetooth: Generally meets my requirements, but the data transfer rate is far too low, being an order of magnitude below WiFi.

WiFi Direct: I did not actually try this, because I could not find a working way to have the PC accept the WiFi Direct connection. It promises exceptional speeds, but there are no modules or libraries that I've found that would help me enable it.

SD Card: The lowest of the low tech methods, just save everything to the SD card and have the client take the card out and plug it into their computer. Would hands-down go with this, except for the need for a SD card port (guaranteed neither for the client's computer or their phone).

The obvious: manual drag and dropping required files from the phone to the PC. This is how we do things now, and it's fraught with problems. Text files have the last 15-20 characters cut off from them, for some reason, during transfer; many of the photos on the device do not initially show up due to the known issue with MTP not forcing a media rescan on the phone; and users sometimes drop files in the wrong places, leading to unnecessary support calls. This method surely needs to be replaced.

I've been searching for a solution for this problem for two months now, and it's hard to believe that nobody has had a use case for Android like this. It's been made clear that USB communication between Android and PC is non-trivial, I'm sure there are programs that do that, Windows to start off with. Yet try as I might, there are no solutions that work out of the box. Has anyone dealt with an issue like this before? What were your solutions? How would you approach this problem?

like image 887
sg.cc Avatar asked Jul 19 '18 14:07

sg.cc


People also ask

CAN Node JS handle large data?

The Node. js stream feature makes it possible to process large data continuously in smaller chunks without keeping it all in memory. In other words, you can use streams to read from or write to a source continuously instead of using the traditional method of processing all of it at once.

Is Node JS good for heavy computation?

js receives a CPU bound task: Whenever a heavy request comes to the event loop, Node. js would set all the CPU available to process it first, and then answer other requests queued. That results in slow processing and overall delay in the event loop, which is why Node. js is not recommended for heavy computation.

Can Node JS be used for Android?

You can use Node. js for Mobile Apps. It works on Android devices and simulators, with pre-built binaries for armeabi-v7a, x86, arm64-v8a, x86_64. It also works on iOS, though that's outside the scope of this question.


1 Answers

When downloading large files on Android it is recommended to use Download Manager, it supports a lot of native features like the progress bar, and notification, also the download task will be handled by OS, and more efficient.

React Native Fetch Blob library support this feature

like image 185
Alex Lévy Avatar answered Nov 15 '22 20:11

Alex Lévy