Nested list of dictionary with nested list of dictionary into a Pandas dataframe

Question

I need help with converting a nested list of dictionaries with a nested list of dictionaries inside of it to a dataframe. At the end, I want something that looks like (the dots are for other columns in between):

    id       | isbn     | isbn13      | .... | average_rating|
    30278752 |1594634025|9781594634024| .... |3.92           |
    34006942 |1501173219|9781501173219| .... |4.33           |

    review_stat =[{'books': [{'id': 30278752,
                              'isbn': '1594634025',
                              'isbn13': '9781594634024',
                              'ratings_count': 4832,
                              'reviews_count': 8435,
                              'text_reviews_count': 417,
                              'work_ratings_count': 2081902,
                              'work_reviews_count': 3313007,
                              'work_text_reviews_count': 109912,
                             'average_rating': '3.92'}]},
                 {'books': [{'id': 34006942,
                             'isbn': '1501173219',
                             'isbn13': '9781501173219',
                             'ratings_count': 4373,
                             'reviews_count': 10741,
                             'text_reviews_count': 565,
                             'work_ratings_count': 1005504,
                             'work_reviews_count': 2142280,
                             'work_text_reviews_count': 75053,
                             'average_rating': '4.33'}]}]

Answer 1

If you key is always books

pd.concat([pd.DataFrame(i['books']) for i in review_stat])

         id        isbn         isbn13  ratings_count  reviews_count  text_reviews_count  work_ratings_count  work_reviews_count  work_text_reviews_count average_rating
0  30278752  1594634025  9781594634024           4832           8435                 417             2081902             3313007                   109912           3.92
0  34006942  1501173219  9781501173219           4373          10741                 565             1005504             2142280                    75053           4.33

You can always reset the index if you need

Answer 2

You can also use json_normalize here:

df = pd.json_normalize(review_stat, 'books')

[out]

         id        isbn  ... work_text_reviews_count  average_rating
0  30278752  1594634025  ...                  109912            3.92
1  34006942  1501173219  ...                   75053            4.33