Creating a nested dictionary from pandas dataframe

Question

I am trying to create a nested dictionary from a pandas dataframe.

I have this dataframe:

# this code should reproduce the example dataframe below:
df = pd.DataFrame({'ScID.xml': {0: '0006-****(****)050[****:ayfb]2.3.co.xml',
  1: '0006-****(****)050[****:smihds]2.3.co.xml',
  2: '0006-****(****)050[****:gissad]2.3.co.xml'},
 'topic1': {0: 7.26744186046513e-06, 1: 0.0006479109, 2: 3.413e-06},
 'topic2': {0: 7.26744186046513e-06, 1: 0.0091339857, 2: 3.413e-06},
 'topic3': {0: 7.26744186046513e-06, 1: 2.79485746226941e-06, 2: 3.413e-06}})

# example dataframe:
                                    ScID.xml    topic1    topic2    topic3
0    0006-****(****)050[****:ayfb]2.3.co.xml  0.000007  0.000007  0.000007
1  0006-****(****)050[****:smihds]2.3.co.xml  0.000648  0.009134  0.000003
2  0006-****(****)050[****:gissad]2.3.co.xml  0.000003  0.000003  0.000003

I would like to produce a nested dictionary like this:

new_dict = {
   'topic1': {'0006-****(****)050[****:ayfb]2.3.co.xml': 0.000007,
   '0006-****(****)050[****:smihds]2.3.co.xml': 0.000648,
   '0006-****(****)050[****:gissad]2.3.co.xml': 0.000003},

   'topic2': {'0006-****(****)050[****:ayfb]2.3.co.xml': 0.000007,
   '0006-****(****)050[****:smihds]2.3.co.xml': 0.009134,
   '0006-****(****)050[****:gissad]2.3.co.xml': 0.000003},

   'topic3': {'0006-****(****)050[****:ayfb]2.3.co.xml': 0.000007,
   '0006-****(****)050[****:smihds]2.3.co.xml': 0.000003,
   '0006-****(****)050[****:gissad]2.3.co.xml': 0.000003}
}

Where topicX are the keys and ScID.xmls are the subkeys.

Answer 1

You can use set_index followed by pd.DataFrame.to_dict.

res = df.set_index('ScID.xml').to_dict(orient='dict')

{'topic1': {'0006-****(****)050[****:ayfb]2.3.co.xml': 7.2674418604651302e-06,
  '0006-****(****)050[****:gissad]2.3.co.xml': 3.4130000000000002e-06,
  '0006-****(****)050[****:smihds]2.3.co.xml': 0.0006479109},
 'topic2': {'0006-****(****)050[****:ayfb]2.3.co.xml': 7.2674418604651302e-06,
  '0006-****(****)050[****:gissad]2.3.co.xml': 3.4130000000000002e-06,
  '0006-****(****)050[****:smihds]2.3.co.xml': 0.0091339856999999997},
 'topic3': {'0006-****(****)050[****:ayfb]2.3.co.xml': 7.2674418604651302e-06,
  '0006-****(****)050[****:gissad]2.3.co.xml': 3.4130000000000002e-06,
  '0006-****(****)050[****:smihds]2.3.co.xml': 2.79485746226941e-06}}

Answer 2

Use DataFrame.set_index with DataFrame.to_dict, parameter orient='dict' is by default, so omited:

d = df.set_index('ScID.xml').to_dict()

print (d)

{'topic1': {'0006-****(****)050[****:ayfb]2.3.co.xml': 7.26744186046513e-06, 
            '0006-****(****)050[****:smihds]2.3.co.xml': 0.0006479109, 
            '0006-****(****)050[****:gissad]2.3.co.xml': 3.413e-06}, 
'topic2': {'0006-****(****)050[****:ayfb]2.3.co.xml': 7.26744186046513e-06, 
           '0006-****(****)050[****:smihds]2.3.co.xml': 0.0091339857, 
          '0006-****(****)050[****:gissad]2.3.co.xml': 3.413e-06}, 
'topic3': {'0006-****(****)050[****:ayfb]2.3.co.xml': 7.26744186046513e-06, 
           '0006-****(****)050[****:smihds]2.3.co.xml': 2.79485746226941e-06, 
           '0006-****(****)050[****:gissad]2.3.co.xml': 3.413e-06}}