Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Convert numpy type to python

I have a list of dicts in the following form that I generate from pandas. I want to convert it to a json format.

list_val = [{1.0: 685}, {2.0: 8}] output = json.dumps(list_val) 

However, json.dumps throws an error: TypeError: 685 is not JSON serializable

I am guessing it's a type conversion issue from numpy to python(?).

However, when I convert the values v of each dict in the array using np.int32(v) it still throws the error.

EDIT: Here's the full code

            new = df[df[label] == label_new]              ks_dict = json.loads(content)             ks_list = ks_dict['variables']             freq_counts = []              for ks_var in ks_list:                      freq_var = dict()                     freq_var["name"] = ks_var["name"]                     ks_series = new[ks_var["name"]]                     temp_df = ks_series.value_counts().to_dict()                     freq_var["new"] = [{u: np.int32(v)} for (u, v) in temp_df.iteritems()]                                 freq_counts.append(freq_var)             out = json.dumps(freq_counts) 
like image 236
ubh Avatar asked Nov 20 '14 21:11

ubh


People also ask

How do you convert Dtype to Python?

We have a method called astype(data_type) to change the data type of a numpy array. If we have a numpy array of type float64, then we can change it to int32 by giving the data type to the astype() method of numpy array.

Can you convert NumPy array to list in Python?

We can use NumPy np. array tolist() function to convert an array to a list. If the array is multi-dimensional, a nested list is returned. For a one-dimensional array, a list with the array elements is returned.

What is NumPy data type in Python?

Complex number, represented by two 64-bit floats (real and imaginary components) NumPy numerical types are instances of dtype (data-type) objects, each having unique characteristics. The dtypes are available as np. bool_, np. float32, etc.


1 Answers

It looks like you're correct:

>>> import numpy >>> import json >>> json.dumps(numpy.int32(685)) Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "/usr/lib/python2.7/json/__init__.py", line 243, in dumps     return _default_encoder.encode(obj)   File "/usr/lib/python2.7/json/encoder.py", line 207, in encode     chunks = self.iterencode(o, _one_shot=True)   File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode     return _iterencode(o, 0)   File "/usr/lib/python2.7/json/encoder.py", line 184, in default     raise TypeError(repr(o) + " is not JSON serializable") TypeError: 685 is not JSON serializable 

The unfortunate thing here is that numpy numbers' __repr__ doesn't give you any hint about what type they are. They're running around masquerading as ints when they aren't (gasp). Ultimately, it looks like json is telling you that an int isn't serializable, but really, it's telling you that this particular np.int32 (or whatever type you actually have) isn't serializable. (No real surprise there -- No np.int32 is serializable). This is also why the dict that you inevitably printed before passing it to json.dumps looks like it just has integers in it as well.

The easiest workaround here is probably to write your own serializer1:

class MyEncoder(json.JSONEncoder):     def default(self, obj):         if isinstance(obj, numpy.integer):             return int(obj)         elif isinstance(obj, numpy.floating):             return float(obj)         elif isinstance(obj, numpy.ndarray):             return obj.tolist()         else:             return super(MyEncoder, self).default(obj) 

You use it like this:

json.dumps(numpy.float32(1.2), cls=MyEncoder) json.dumps(numpy.arange(12), cls=MyEncoder) json.dumps({'a': numpy.int32(42)}, cls=MyEncoder) 

etc.

1Or you could just write the default function and pass that as the defaut keyword argument to json.dumps. In this scenario, you'd replace the last line with raise TypeError, but ... meh. The class is more extensible :-)

like image 193
mgilson Avatar answered Oct 01 '22 11:10

mgilson