Pandas json_normalize and null values in JSON

Question

I have this sample JSON

{
    "name":"John",
    "age":30,
    "cars": [
        { "name":"Ford", "models":[ "Fiesta", "Focus", "Mustang" ] },
        { "name":"BMW", "models":[ "320", "X3", "X5" ] },
        { "name":"Fiat", "models":[ "500", "Panda" ] }
    ]
 }

When I need to convert JSON to pandas DataFrame I use following code

import json
from pandas.io.json import json_normalize
from pprint import pprint

with open('example.json', encoding="utf8") as data_file:
    data = json.load(data_file)
normalized = json_normalize(data['cars'])

This code works well but in the case of some empty cars (null values) I'm not possible to normalize_json.

Example of json

{
    "name":"John",
    "age":30,
    "cars": [
        { "name":"Ford", "models":[ "Fiesta", "Focus", "Mustang" ] },
        null,
        { "name":"Fiat", "models":[ "500", "Panda" ] }
    ]
 }

Error that was thrown

AttributeError: 'NoneType' object has no attribute 'keys'

I tried to ignore errors in json_normalize, but didn't help

normalized = json_normalize(data['cars'], errors='ignore')

How should I handle null values in JSON?

Answer 1

You can fill cars with empty dicts to prevent this error

data['cars'] = data['cars'].apply(lambda x: {} if pd.isna(x) else x)