Nested dictionary from large pandas dataframe

Question

I am trying to create a nested dictionary from a pandas dataframe.

I have this dataframe:

# this code should reproduce the example dataframe below:
df = pd.DataFrame({'ScID.xml': {0: '0006-****(****)050[****:ayfb]2.3.co.xml',
  1: '0006-****(****)050[****:smihds]2.3.co.xml',
  2: '0006-****(****)050[****:gissad]2.3.co.xml'},
 'topic1': {0: 7.26744186046513e-06, 1: 0.0006479109, 2: 3.413e-06},
 'topic2': {0: 7.26744186046513e-06, 1: 0.0091339857, 2: 3.413e-06},
 'topic3': {0: 7.26744186046513e-06, 1: 2.79485746226941e-06, 2: 3.413e-06}})

# example dataframe:
                                    ScID.xml    topic1    topic2    topic3
0    0006-****(****)050[****:ayfb]2.3.co.xml  0.000007  0.000007  0.000007
1  0006-****(****)050[****:smihds]2.3.co.xml  0.000648  0.009134  0.000003
2  0006-****(****)050[****:gissad]2.3.co.xml  0.000003  0.000003  0.000003

I would like to produce a nested dictionary like this:

new_dict = {
   '0006-****(****)050[****:ayfb]2.3.co.xml': {'topic1': 0.000007,
   'topic2': 0.000007,
   'topic3': 0.000007},
   '0006-****(****)050[****:smihds]2.3.co.xml': {'topic1': 0.000648,
   'topic2': 0.009134,
   'topic3': 0.000003},
   '0006-****(****)050[****:gissad]2.3.co.xml': {'topic1': 0.000003,
   'topic2': 0.000003,
   'topic3': 0.000003}
}

I have tried:

new_dict = df.set_index('ScID.xml').T.to_dict()

but that returns a dictionary in the following format:

{0: {'ScID.xml': 0006-****(****)050[****:ayfb]2.3.co.xml,
'topic1': 0.000007,
'topic2': 0.000007,
'topic3': 0.000007}
}

Answer 1

Try setting the keys as the index, then using index orientation in to_dict:

>>> df.set_index('ScID.xml').to_dict(orient='index')
{'0006-****(****)050[****:ayfb]2.3.co.xml': {'topic1': 7.2674418604651302e-06,
  'topic2': 7.2674418604651302e-06,
  'topic3': 7.2674418604651302e-06},
 '0006-****(****)050[****:gissad]2.3.co.xml': {'topic1': 3.4130000000000002e-06,
  'topic2': 3.4130000000000002e-06,
  'topic3': 3.4130000000000002e-06},
 '0006-****(****)050[****:smihds]2.3.co.xml': {'topic1': 0.0006479109,
  'topic2': 0.0091339856999999997,
  'topic3': 2.79485746226941e-06}}