Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Making numpy arrays JSON serializable

I have a program which handles data in the form of NumPy arrays, which needs to be stored in JSON form (to later pass it to another program to visualize the data). When I try the following:

my_array = np.array([3, 4, 5])
json.dumps(my_array)

I get an error message reading

TypeError: array([3, 4, 5]) is not JSON serializable

So it turns out that arrays are not serializable. I hoped to solve this by converting the arrays into ordinary lists, but if I try

my_array = np.array([3, 4, 5])
my_list = list(my_array)
json.dumps(my_list)

I just get an error reading

TypeError: 3 is not JSON serializable

(That 3 appears to be because '3' is the first element of the list)

Even stranger, this error persists when I try to reconstruct a list from scratch:

def plain_list(ls):
  pl = []
  for element in ls:
    pl.append(element)
  return pl

my_array = np.array([3, 4, 5])
my_list = plain_list(my_array)
json.dumps(my_list)

still returns

TypeError: 3 is not JSON serializable

I have two questions:

  1. How is it that even that last one doesn't work? Shouldn't it have 'forgotten' everything about having ever been a NumPy array? Are integers in NumPy arrays not really the same objects as ordinary integers, and do they carry some special invisible property with them that persists even outside of the array?
  2. How can I effectively store arrays in JSON form (no matter how convoluted the solution may need to be)?
like image 546
Drubbels Avatar asked Jan 17 '18 21:01

Drubbels


People also ask

Are NumPy arrays serializable?

As far as I know you can not simply serialize a numpy array with any data type and any dimension...but you can store its data type, dimension and information in a list representation and then serialize it using JSON.

Is an array JSON serializable?

NumPy array is not JSON serializable.

How do I make my Python object JSON serializable?

Use toJSON() Method to make class JSON serializable So we don't need to write custom JSONEncoder. This new toJSON() serializer method will return the JSON representation of the Object. i.e., It will convert custom Python Object to JSON string.

Is Python set JSON serializable?

If you try to convert the Set to json, you will get this error: TypeError: Object of type set is not JSON serializable. This is because the inbuilt Python json module can only handle primitives data types with a direct JSON equivalent and not complex data types like Set.


1 Answers

That 3 is a NumPy integer that displays like a regular Python int, but isn't one. Use tolist to get a list of ordinary Python ints:

json.dumps(my_array.tolist())

This will also convert multidimensional arrays to nested lists, so you don't have to deal with a 3-layer list comprehension for a 3-dimensional array.

like image 64
user2357112 supports Monica Avatar answered Oct 21 '22 05:10

user2357112 supports Monica