How to merge two json file in Python?

Question

I'm trying to do a python script that merge 2 json files for example:

First file: test.json

[
  {
    "user_id": "273631610",
    "item_id": "0829482",
    "rating": "7"
  },
  {
    "user_id": "40688800",
    "item_id": "1343092",
    "rating": "8"
  },
  {
    "user_id": "395680865",
    "item_id": "1453405",
    "rating": "8"
  }
]

Second file: test_userz.json

[
{"contributors": null, "truncated": false, "text": "", "in_reply_to_status_id": null, "id": 421040281296052225, "favorite_count": 0, "source": "<a href=\"http://itunes.apple.com/us/app/imdb-movies-tv/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [{"indices": [23, 28], "text": "IMDb"}], "urls": [{"url": "5E1fLOxB", "indices": [29, 51], "expanded_url": "http://www.imdb.com/title/tt0829482", "display_url": "imdb.com/title/tt0829482"}]}, "in_reply_to_screen_name": null, "id_str": "421040281296052225", "retweet_count": 0, "in_reply_to_user_id": null, "favorited": false, "user": {"follow_request_sent": false, "profile_use_background_image": true, "id": 273631610, "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/378800000723050633/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg", "profile_sidebar_fill_color": "DDEEF6", "is_translator": false, "geo_enabled": true, "entities": {"description": {"urls": []}}, "followers_count": 330, "protected": false, "location": "southend on sea", "default_profile_image": false, "id_str": "273631610", "lang": "en", "utc_offset": null, "statuses_count": 897, "description": "", "friends_count": 973, "profile_link_color": "0084B4", "profile_image_url": "http://pbs.twimg.com/profile_images/378800000723050633/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg", "notifications": false, "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png", "profile_background_color": "C0DEED", "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png", "name": "Ben Morris", "is_translation_enabled": false, "profile_background_tile": false, "favourites_count": 12, "screen_name": "benyere", "url": null, "created_at": "Mon Mar 28 21:31:52 +0000 2011", "contributors_enabled": false, "time_zone": null, "profile_sidebar_border_color": "C0DEED", "default_profile": true, "following": false, "listed_count": 0}, "geo": null, "in_reply_to_user_id_str": null, "possibly_sensitive": false, "lang": "en", "created_at": "Wed Jan 08 22:06:40 +0000 2014", "in_reply_to_status_id_str": null, "place": null},
{"contributors": null, "truncated": false, "text": "", "in_reply_to_status_id": null, "id": 421040870931320833, "favorite_count": 0, "source": "<a href=\"http://www.apple.com\" rel=\"nofollow\">iOS</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [{"indices": [31, 36], "text": "IMDb"}], "urls": [{"url": "dTFeexXDk", "indices": [37, 59], "expanded_url": "http://www.imdb.com/title/tt1343092", "display_url": "imdb.com/title/tt1343092"}]}, "in_reply_to_screen_name": null, "id_str": "421040870931320833", "retweet_count": 0, "in_reply_to_user_id": null, "favorited": false, "user": {"follow_request_sent": false, "profile_use_background_image": false, "id": 40688800, "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/378800000807375392/2ee9bbee70e09ac097f8415794877e7c_normal.jpeg", "profile_sidebar_fill_color": "FFFFFF", "is_translator": false, "geo_enabled": true, "entities": {"description": {"urls": []}}, "followers_count": 230, "protected": false, "location": "Alexandria", "default_profile_image": false, "id_str": "40688800", "lang": "en", "utc_offset": 7200, "statuses_count": 8914, "description": "#TedxAlexandriaU head. resp of #emshi_fe_7artak in #alex Day dreamer. #medstudent by accident. I speak sarcasm.", "friends_count": 722, "profile_link_color": "0099CC", "profile_image_url": "http://pbs.twimg.com/profile_images/378800000807375392/2ee9bbee70e09ac097f8415794877e7c_normal.jpeg", "notifications": false, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/778453690/27087117198cdf1d18bb86a36140de62.jpeg", "profile_background_color": "FFF04D", "profile_banner_url": "https://pbs.twimg.com/profile_banners/40688800/1378766815", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/778453690/27087117198cdf1d18bb86a36140de62.jpeg", "name": "NOUR", "is_translation_enabled": false, "profile_background_tile": true, "favourites_count": 324, "screen_name": "Nour_Ayman", "url": null, "created_at": "Sun May 17 16:38:59 +0000 2009", "contributors_enabled": false, "time_zone": "Cairo", "profile_sidebar_border_color": "FFFFFF", "default_profile": false, "following": false, "listed_count": 2}, "geo": null, "in_reply_to_user_id_str": null, "possibly_sensitive": false, "lang": "en", "created_at": "Wed Jan 08 22:09:00 +0000 2014", "in_reply_to_status_id_str": null, "place": null},
{"contributors": null, "truncated": false, "text": "", "in_reply_to_status_id": null, "id": 421041098287755264, "favorite_count": 0, "source": "<a href=\"http://www.apple.com\" rel=\"nofollow\">iOS</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [{"indices": [34, 39], "text": "IMDb"}], "urls": [{"url": "I2HqieyDnn", "indices": [40, 62], "expanded_url": "http://www.imdb.com/title/tt1453405", "display_url": "imdb.com/title/tt1453405"}]}, "in_reply_to_screen_name": null, "id_str": "421041098287755264", "retweet_count": 0, "in_reply_to_user_id": null, "favorited": false, "user": {"follow_request_sent": false, "profile_use_background_image": true, "id": 395680865, "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/3130997221/4a981c8594f3c9ec127542ae40bac4ce_normal.jpeg", "profile_sidebar_fill_color": "DDEEF6", "is_translator": false, "geo_enabled": false, "entities": {"description": {"urls": []}}, "followers_count": 9, "protected": false, "location": "Denmark", "default_profile_image": false, "id_str": "395680865", "lang": "en", "utc_offset": 3600, "statuses_count": 1557, "description": "On basketball, film and other really cool things.", "friends_count": 27, "profile_link_color": "0084B4", "profile_image_url": "http://pbs.twimg.com/profile_images/3130997221/4a981c8594f3c9ec127542ae40bac4ce_normal.jpeg", "notifications": false, "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png", "profile_background_color": "C0DEED", "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png", "name": "Kristian Meller", "is_translation_enabled": false, "profile_background_tile": false, "favourites_count": 5, "screen_name": "kristianmeller", "url": null, "created_at": "Sat Oct 22 02:18:30 +0000 2011", "contributors_enabled": false, "time_zone": "Berlin", "profile_sidebar_border_color": "C0DEED", "default_profile": true, "following": false, "listed_count": 0}, "geo": null, "in_reply_to_user_id_str": null, "possibly_sensitive": false, "lang": "en", "created_at": "Wed Jan 08 22:09:55 +0000 2014", "in_reply_to_status_id_str": null, "place": null}
]

I would like to merge these into a single array like this:

[
{"user_id":"273631610","item_id":"0829482","rating":"7","contributors":null,"truncated":false,"text":"","in_reply_to_status_id":null,"id":421040281296052225,"favorite_count":0,"source":"<a href=\"http:\/\/itunes.apple.com\/us\/app\/imdb-movies-tv\/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS<\/a>","retweeted":false,"coordinates":null,"entities":{"symbols":[],"user_mentions":[],"hashtags":[{"indices":[23,28],"text":"IMDb"}],"urls":[{"url":"/bk5E1fLOxB","indices":[29,51],"expanded_url":"http:\/\/www.imdb.com\/title\/tt0829482","display_url":"imdb.com\/title\/tt0829482"}]},"in_reply_to_screen_name":null,"id_str":"421040281296052225","retweet_count":0,"in_reply_to_user_id":null,"favorited":false,"user":{"follow_request_sent":false,"profile_use_background_image":true,"id":273631610,"verified":false,"profile_text_color":"333333","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","profile_sidebar_fill_color":"DDEEF6","is_translator":false,"geo_enabled":true,"entities":{"description":{"urls":[]}},"followers_count":330,"protected":false,"location":"southend on sea","default_profile_image":false,"id_str":"273631610","lang":"en","utc_offset":null,"statuses_count":897,"description":"","friends_count":973,"profile_link_color":"0084B4","profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","notifications":false,"profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","name":"Ben Morris","is_translation_enabled":false,"profile_background_tile":false,"favourites_count":12,"screen_name":"benyere","url":null,"created_at":"Mon Mar 28 21:31:52 +0000 2011","contributors_enabled":false,"time_zone":null,"profile_sidebar_border_color":"C0DEED","default_profile":true,"following":false,"listed_count":0},"geo":null,"in_reply_to_user_id_str":null,"possibly_sensitive":false,"lang":"en","created_at":"Wed Jan 08 22:06:40 +0000 2014","in_reply_to_status_id_str":null,"place":null},
{"user_id":"40688800","item_id":"1343092","rating":"8","contributors":null,"truncated":false,"text":"","in_reply_to_status_id":null,"id":421040281296052225,"favorite_count":0,"source":"<a href=\"http:\/\/itunes.apple.com\/us\/app\/imdb-movies-tv\/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS<\/a>","retweeted":false,"coordinates":null,"entities":{"symbols":[],"user_mentions":[],"hashtags":[{"indices":[23,28],"text":"IMDb"}],"urls":[{"url":"/bk5E1fLOxB","indices":[29,51],"expanded_url":"http:\/\/www.imdb.com\/title\/tt0829482","display_url":"imdb.com\/title\/tt0829482"}]},"in_reply_to_screen_name":null,"id_str":"421040281296052225","retweet_count":0,"in_reply_to_user_id":null,"favorited":false,"user":{"follow_request_sent":false,"profile_use_background_image":true,"id":273631610,"verified":false,"profile_text_color":"333333","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","profile_sidebar_fill_color":"DDEEF6","is_translator":false,"geo_enabled":true,"entities":{"description":{"urls":[]}},"followers_count":330,"protected":false,"location":"southend on sea","default_profile_image":false,"id_str":"273631610","lang":"en","utc_offset":null,"statuses_count":897,"description":"","friends_count":973,"profile_link_color":"0084B4","profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","notifications":false,"profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","name":"Ben Morris","is_translation_enabled":false,"profile_background_tile":false,"favourites_count":12,"screen_name":"benyere","url":null,"created_at":"Mon Mar 28 21:31:52 +0000 2011","contributors_enabled":false,"time_zone":null,"profile_sidebar_border_color":"C0DEED","default_profile":true,"following":false,"listed_count":0},"geo":null,"in_reply_to_user_id_str":null,"possibly_sensitive":false,"lang":"en","created_at":"Wed Jan 08 22:06:40 +0000 2014","in_reply_to_status_id_str":null,"place":null},
{"user_id":"3956800865","item_id":"1453405","rating":"8","contributors":null,"truncated":false,"text":"","in_reply_to_status_id":null,"id":421040281296052225,"favorite_count":0,"source":"<a href=\"http:\/\/itunes.apple.com\/us\/app\/imdb-movies-tv\/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS<\/a>","retweeted":false,"coordinates":null,"entities":{"symbols":[],"user_mentions":[],"hashtags":[{"indices":[23,28],"text":"IMDb"}],"urls":[{"url":"\/bk5E1fLOxB","indices":[29,51],"expanded_url":"http:\/\/www.imdb.com\/title\/tt0829482","display_url":"imdb.com\/title\/tt0829482"}]},"in_reply_to_screen_name":null,"id_str":"421040281296052225","retweet_count":0,"in_reply_to_user_id":null,"favorited":false,"user":{"follow_request_sent":false,"profile_use_background_image":true,"id":273631610,"verified":false,"profile_text_color":"333333","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","profile_sidebar_fill_color":"DDEEF6","is_translator":false,"geo_enabled":true,"entities":{"description":{"urls":[]}},"followers_count":330,"protected":false,"location":"southend on sea","default_profile_image":false,"id_str":"273631610","lang":"en","utc_offset":null,"statuses_count":897,"description":"","friends_count":973,"profile_link_color":"0084B4","profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","notifications":false,"profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","name":"Ben Morris","is_translation_enabled":false,"profile_background_tile":false,"favourites_count":12,"screen_name":"benyere","url":null,"created_at":"Mon Mar 28 21:31:52 +0000 2011","contributors_enabled":false,"time_zone":null,"profile_sidebar_border_color":"C0DEED","default_profile":true,"following":false,"listed_count":0},"geo":null,"in_reply_to_user_id_str":null,"possibly_sensitive":false,"lang":"en","created_at":"Wed Jan 08 22:06:40 +0000 2014","in_reply_to_status_id_str":null,"place":null}
]

To do that I did the following code:

import json

with open("test.json") as fin1:
    data1 = json.load(fin1)
with open("test_userz.json") as fin2:
    data2 = json.load(fin2)
data1.update(data2)
with open("merged.json", "w") as fout:
    json.dump(data1, fout)

but in runtime i see this error :

Traceback (most recent call last):
File "json/merge2jsons.py", line 7, in <module>
data1.update(data2)
AttributeError: 'list' object has no attribute 'update'

How can i do this merge ?

Answer 1

The error you are facing is that your data1 upon loading from your file is going to be a list. You are trying to call the dictionary update on it, which is incorrect.

To help provide an example of your error, look at this:

>>> [].update('stuff')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'update'

I tried calling update on my list. There is no update method on the list. If I do this for a dictionary, however:

>>> d = {'a': 1}
>>> d.update({'b': 2})
>>> d
{'b': 2, 'a': 1}

As you can see, the update method exists, and works fine.

So, what you want to do is iterate over one of the list of dictionaries, and just update along the way. Since you are confident in your data being sequential and of equal length, this can be simply done by:

with open('f1.txt') as f1, open('f2.txt') as f2:
    first_list = json.load(f1)
    second_list = json.load(f2)

for i, v in enumerate(first_list):
    second_list[i].update(v)

So, what is happening here is that by using the enumerate, you will have an incremental count representing the index of your list to easily access each dictionary. With this, simply update the dictionary of each dict in second_list with the dictionary you get from each iteration of first_list

Answer 2

To merge dictionaries that are at the same index in two lists:

merged = [{**d1, **d2} for d1, d2 in zip(data1, data2)]