ValueError: Conflicting metadata name name, need distinguishing prefix in pandas

Question

I am getting a ValueError when I execute the following

df = pd.DataFrame.from_dict(
                  json_normalize(
                                 data,
                                 'protocol_parameters',
                                 [['status','status'], 
                                  'auto_discovered', 
                                  'average_eps',
                                  'creation_date',
                                  'description',
                                  'last_event_time',
                                  'name']
                                ), orient='columns')

Data Object used in the code

{
        "protocol_parameters": [
            {
                "name": "identifier",
                "value": "x.x.x.x"
            },
            {
                "name": "incomingPayloadEncoding",
                "value": "UTF-8"
            }
        ],
        "description": "LinuxServer device",
        "average_eps": 0,
        "creation_date": 0,
        "name": "LinuxServer @ x.x.x.x",
        "auto_discovered": true,
        "last_event_time": 1535539535018,
        "status": {
            "status": "SUCCESS"
        }
    },

Error Message ( I guess because name field is in 'protocol_parameters' and also a separate object. But I am unable to fix this issue

Traceback (most recent call last):
  File "D:\Qradar\python\LogSources.py", line 31, in <module>
    df = pd.DataFrame.from_dict(json_normalize(data,'protocol_parameters',[['status','status'],'auto_discovered','average_eps','creation_date','description','last_event_time','name']), orient='columns')
  File "D:\VM\python\lib\site-packages\pandas\io\json\normalize.py", line 262, in json_normalize
    'need distinguishing prefix ' % k)
ValueError: Conflicting metadata name name, need distinguishing prefix 

Answer 1

You are close all you need to do is add record_prefix to handle name being used twice. Set the prefix to whatever string you want:

json_normalize(data,
               'protocol_parameters',
               [['status','status'], 
               'auto_discovered', 
               'average_eps',
               'creation_date',
               'description',
               'last_event_time',
               'name'], record_prefix='_'
                )

out:

_name   _value  status.status   auto_discovered average_eps creation_date   description last_event_time name
0   identifier  x.x.x.x SUCCESS True    0   0   LinuxServer device  1535539535018   LinuxServer @ x.x.x.x
1   incomingPayloadEncoding UTF-8   SUCCESS True    0   0   LinuxServer device  1535539535018   LinuxServer @ x.x.x.x