Loading numpy array from http response without saving a file

Question

I have a bunch of files containing numpy arrays at some url (e.g., https://my_url/my_np_file.npy) and I am trying to load them in my computer.

If I download the file manually, I can properly load the numpy array using np.load('file_path'). If I take the url reponse (using the code below), save the content to a file and then use np.load(), it also works.

response, content = http.request('https://my_url/my_np_file.npy')

If I try to load the array from the content string, I get the error bellow. This is likely because the np.load is interpreting the string input as a name for the file, rather than the data itself.

File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 370, in load fid = open(file, "rb") TypeError: file() argument 1 must be encoded string without null bytes, not str

Is there any way to load the array without having to save a file?

Answer 1

You are missing io.BytesIO to make the string appear like a file object to np.load!

The following is what you're looking for:

import requests
import io

response = requests.get('https://my_url/my_np_file.npy')
response.raise_for_status()
data = np.load(io.BytesIO(response.content))  # Works!