Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

TypeError: Object of type 'ndarray' is not JSON serializable

Tags:

python

json

knownEmbeddings = []
knownNames = []

for (i, imagePath) in enumerate(imagePaths):
    ## SOME CODE
    knownNames.append(name)
    knownEmbeddings.append(vec.flatten())


data = {"embeddings": knownEmbeddings, "names": knownNames}

f = open('file.json', "wb")
f.write(json.dumps(data, indent=4))
f.close()

Here is the data looks like:

{'embeddings': [array([ 2.23568859e-04, -4.08176295e-02, -1.56606492e-02, -1.40566211e-02,
        5.53448219e-04,  1.34807974e-01,  2.10583732e-02, -7.99260102e-03,
        8.04360434e-02,  2.51036473e-02, -2.45967298e-03,  8.73192959e-03,
        1.08047323e-02,  8.02712217e-02,  6.31465465e-02,  9.41963419e-02],
      dtype=float32), array([-5.54675907e-02,  1.19409459e-02, -3.03599555e-02, -2.86714472e-02,
        6.26528710e-02,  1.25348523e-01, -2.16291733e-02, -4.60545160e-02,
        6.25465512e-02, -7.61162862e-02,  4.28330414e-02,  8.57844874e-02,
        3.75184380e-02, -8.10878351e-02, -8.96525383e-02,  8.15552175e-02,
       -9.75750014e-02, -8.24848488e-02,  9.30746570e-02,  1.71318889e-01,
        1.00642473e-01,  5.39120510e-02,  1.12627009e-02,  1.40678780e-02,
       -4.41719554e-02,  1.03237763e-01,  4.38372791e-02,  7.53327608e-02],
      dtype=float32), array([-0.03736538, -0.0485549 , -0.0976112 , -0.06195155,  0.00269726,
        0.07389018, -0.07325964,  0.06653353, -0.04690087, -0.02606474,
        0.03597135,  0.06904668,  0.02198682, -0.06437466, -0.04554454,
        0.01083081, -0.06293067,  0.07047471,  0.02824293, -0.15522538,
       -0.01900602,  0.10689866, -0.07408814, -0.0419201 ,  0.10475922,
        0.04784475, -0.09085421, -0.20360689,  0.08321641,  0.08441921,
        0.01941148,  0.03566081, -0.05956643,  0.005247  , -0.03989819,
        0.02512971,  0.00458561,  0.13706829], dtype=float32), array([ 7.87236728e-03,  5.65276742e-02, -7.17918649e-02, -1.84332877e-02,
        1.28411269e-02,  2.85971135e-01,  3.13642109e-03,  2.48481780e-02,
       -9.48547944e-02,  2.89725009e-02,  4.33916636e-02,  9.01726633e-02,
        4.36290540e-02, -1.02897413e-01,  2.22285688e-02, -5.19381762e-02,
        1.52556881e-01, -1.25146270e-01,  3.18806712e-03, -2.51053665e-02,
       -4.36606398e-03,  7.19061792e-02,  4.66747172e-02,  8.13280419e-02],
      dtype=float32), array([ 0.09142991, -0.05100765, -0.09615178, -0.03553161,  0.11363017,
        0.19886988,  0.11280693,  0.0229619 , -0.0220201 , -0.01211688,
        0.07489388,  0.0802715 ,  0.16185616, -0.0904082 ,  0.0025941 ,
        0.12167819, -0.07357537, -0.01442344, -0.01343578,  0.16952834,
        0.03366659, -0.0534111 , -0.01595308,  0.15053654, -0.07398864,
        0.04694209, -0.06523879,  0.01342433], dtype=float32), array([-0.05331187,  0.08159426, -0.01742208,  0.00992642, -0.01155609,
        0.25759327, -0.00505029, -0.09290393,  0.01588799, -0.00478396,
        0.08572742, -0.05053008,  0.05197625,  0.1267016 ,  0.15398905,
        0.13668832, -0.13869229,  0.02502107, -0.04443422, -0.05987623,
        0.14948404,  0.03311499,  0.12621029], dtype=float32), array([ 0.1219558 , -0.0371135 , -0.13762642,  0.00431138,  0.20073804,
        0.09986125,  0.21617071,  0.02764285, -0.1352063 ,  0.02268699,
       -0.04734468,  0.10888206,  0.13558514, -0.00319178,  0.02979032,
        0.03558976, -0.07293532, -0.05351996, -0.02449711,  0.1459181 ,
       -0.00320001,  0.01020296, -0.05007216,  0.05868218, -0.03522768,
       -0.01064874, -0.0732395 , -0.05393502], dtype=float32), array([ 0.10833652,  0.08779355, -0.15162815, -0.03925862,  0.08713786,
        0.2850307 ,  0.13499181,  0.01792248, -0.1405847 ,  0.08626581,
        0.02001712, -0.06957201, -0.00727825,  0.01650161,  0.11886367,
        0.07897119, -0.14108546,  0.03840445,  0.05881708,  0.03361814,
       -0.0106756 , -0.04287936, -0.06621028], dtype=float32), array([ 4.90577929e-02,  9.13119391e-02, -2.76884548e-02, -5.19143604e-02,
        1.50506735e-01,  1.86451554e-01,  9.94046330e-02, -7.73873506e-03,
       -1.91362634e-01,  4.69892733e-02, -5.67045361e-02,  2.81608831e-02,
        5.74332848e-02, -9.09122005e-02,  1.46917075e-01,  4.63287433e-04,
        4.22818065e-02, -2.01395284e-02,  1.31114023e-02, -6.61114752e-02],
      dtype=float32), array([ 1.13848910e-01,  1.16239523e-03, -6.73869327e-02,  8.96331621e-05,
        6.71111122e-02,  2.01299891e-01,  1.76381439e-01,  1.44544961e-02,
       -1.36415318e-01, -3.18108648e-02, -3.51585075e-02,  1.24862537e-01,
        6.54390603e-02, -1.79662079e-01,  8.39038659e-03, -6.52492717e-02,
       -4.79320846e-02, -4.05376814e-02, -1.82695538e-02,  1.35992825e-01,
        6.97307214e-02, -5.41270301e-02,  3.14575769e-02,  2.86752880e-02,
        9.04180668e-03,  3.10734902e-02, -3.88299376e-02, -7.43401796e-02],
      dtype=float32), array([ 0.09236415,  0.05246023, -0.03693461,  0.05469636,  0.05779893,
        0.13331857,  0.21085702, -0.01114039, -0.09325632,  0.07158454,
        0.03167493,  0.13376454,  0.13156445, -0.12092946, -0.02573274,
       -0.05352074,  0.00177706,  0.05248505, -0.07331309,  0.06653137,
       -0.02102634,  0.00347302, -0.19828801, -0.08791062,  0.05434143,
        0.07060813, -0.09335811, -0.04778329, -0.02983012,  0.1595401 ,
        0.01018381, -0.04852933, -0.03336967,  0.02886004,  0.05975606,
        0.0974864 , -0.00946077, -0.06796782], dtype=float32), array([ 7.70701468e-02,  2.34568515e-03, -1.22838043e-01,  6.06604481e-05,
        1.08674861e-01,  2.40898028e-01,  7.23511800e-02, -4.14036550e-02,
       -1.20636895e-01, -2.74732499e-03, -1.84729435e-02,  6.18617162e-02,
        8.97722915e-02, -1.62845016e-01, -1.34318219e-02,  5.31670935e-02,
       -8.27090293e-02, -1.22121066e-01,  1.53016988e-02,  1.22807577e-01,
       -1.36648446e-01,  1.32556446e-02, -8.84201974e-02,  8.29895660e-02,
        5.18502928e-02, -7.32250437e-02,  6.30651340e-02, -9.98577196e-03,
       -4.71815281e-02,  6.02727272e-02, -7.98970908e-02, -2.12689787e-02],
      dtype=float32)], 'names': ['adam', 'adam', 'adam', 'adam', 'adam', 'adam', 'bhalla', 'bhalla', 'bhalla', 'bhalla', 'bhalla', 'bhalla']}

I need to save it in json file. When I debug, the data shows as type dict so I used json.dumps(data) to save it in json file but it is throwing below error:

TypeError: Object of type 'ndarray' is not JSON serializable

How can I resolve it

like image 216
S Andrew Avatar asked Jul 30 '19 10:07

S Andrew


People also ask

How do I serialize a NumPy array to JSON?

Use the cls kwarg of the json. dump() and json. dumps() method to call our custom JSON Encoder, which will convert NumPy array into JSON formatted data. To serialize Numpy array into JSON we need to convert it into a list structure using a tolist() function.

How do you make an array JSON serializable in Python?

Use toJSON() Method to make class JSON serializable So we don't need to write custom JSONEncoder. This new toJSON() serializer method will return the JSON representation of the Object. i.e., It will convert custom Python Object to JSON string. Let' see the example.

Are NumPy arrays serializable?

Python's NumPy array can be used to serialize and deserialize data to and from byte representation.

Which of the following is not JSON serializable?

Conclusion # The Python "TypeError: Object of type function is not JSON serializable" occurs when we try to serialize a function to JSON. To solve the error, make sure to call the function and serialize the object that the function returns.


2 Answers

Adding from previous answers and this post, you can do something like this:

class NumpyEncoder(json.JSONEncoder):
    """ Special json encoder for numpy types """
    def default(self, obj):
        if isinstance(obj, (np.int_, np.intc, np.intp, np.int8,
                            np.int16, np.int32, np.int64, np.uint8,
                            np.uint16, np.uint32, np.uint64)):
            return int(obj)
        elif isinstance(obj, (np.float_, np.float16, np.float32,
                              np.float64)):
            return float(obj)
        elif isinstance(obj, (np.ndarray,)):
            return obj.tolist()
        return json.JSONEncoder.default(self, obj)

Then you can call

import numpy as np
import json
operation = json.dumps(data, cls=NumpyEncoder)

In case you are working with a dictionary and want to save the modified dictionary but receiving a similar error

'TypeError: Object of type intc is not JSON serializable'

you can use

dumped = json.dumps(data, cls=NumpyEncoder)
with open('json_file_path.json', 'a') as f:
    f.write(dumped + '\n') 

The code ensure the save json file does not contain \ splashes as stated in here

like image 173
burhan rashid Avatar answered Nov 05 '22 04:11

burhan rashid


data is a dict indeed, but it contains lists of ndarrays, which need to be serialized too, and are not (natively) serializable.

What you need here is to provide your own JSONEncoder that knows how to deal with ndarray (turning them into lists as suggested by bharatk being the most obvious solution).

like image 38
bruno desthuilliers Avatar answered Nov 05 '22 04:11

bruno desthuilliers