Menu
i am trying to import a JSON file i pull from a URL and send it to mongoDB as is, using the pymongo module.
I have the following code
#!/usr/bin/env python
import sys, urllib2, json, pymongo
from pymongo import MongoClient
myurl = "https://gist.githubusercontent.com/border/775526/raw/b921df18ba00262ab5bba8cadb3c178e1f7748f7/config.json"
response = urllib2.urlopen(myurl)
data = response.read()
connection = MongoClient('mongodb://user:[email protected]:27017/database')
connection.database_names()
db = connection.database
posts = db.posts
post_id = posts.insert_many(data).inserted_id
upon executing this, i get this error raise TypeError("documents must be a non-empty list") TypeError: documents must be a non-empty list
ideally, i want to just be able to pull the json from the url and update the mongoDB as this json file will be updated every week. Thanks
641
To import JSON file you need to follow the following steps: Step 1: Open a command prompt and give command mongod to connect with MongoDB server and don't close this cmd to stay connected to the server. Step 2: Open another command prompt and run the mongo shell. Using the mongo command.
The process to import JSON into MongoDB depends on the operating system and the programming language you are using. However, the key to importing is to access the MongoDB database and parsing the file that you want to import. You can then go through each document sequentially and insert into MongoDB.
The first step to connect python to Atlas is MongoDB cluster setup. Next, create a file named pymongo_test_insert.py in any folder to write pymongo code. You can use any simple text editor like Textpad/Notepad. Use the connection_string to create the mongoclient and get the MongoDB database connection.
You need to convert JSON to Python objects, which PyMongo will then convert to BSON for sending to MongoDB. To convert JSON to Python objects use the "bson.json_util" module included with PyMongo:
from bson import json_util
data = json_util.loads(response.read())
The standard Python json.loads() function works, too, but PyMongo's json_util.loads() handles some MongoDB-specific details better.
139
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
