Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

can't read json file with python. getting type error: json object is 'TextIOWrapper'

I'm trying to read from a json file.

This is how I created the file:

import requests
import json
import time
from pprint import pprint

BASE_URL = "https://www.wikiart.org/en/api/2/UpdatedArtists"
artist_json_data = requests.get(BASE_URL).json()

with open('artistdata.json', 'w') as outfile:
    while artist_json_data['hasMore']:
        print(artist_json_data['paginationToken'])
        url = BASE_URL + "?paginationToken=" +artist_json_data['paginationToken']
        artist_json_data = requests.get(url).json()
        json.dump(artist_json_data, outfile, indent=4)
        time.sleep(1)

This is the beginning of my output:

{
    "data": [
        {
            "id": "57726da5edc2cb3880b4ca54",
            "artistName": "Paul Feeley",
            "url": "paul-feeley",
            "lastNameFirst": "Feeley Paul",
            "birthDay": "/Date(-1893456000000)/",
            "deathDay": "/Date(-126230400000)/",

When I try to read the same file with following code:

from pprint import pprint

with open('artistdata.json', 'r', encoding='utf-8') as data_file:    
    data = json.loads(data_file)
    pprint(data)

I get the error

TypeError: the JSON object must be str, bytes or bytearray, not 'TextIOWrapper'

which I don't understand, because I can open the file in sublime as usual. How can I handle this?

Solved the issue with following code:

the issue was that I mixed dumps and load. Now I am using dump and load

class Wikiart:
    '''Class to access wikiart.org Data'''
    def __init__(self):
        self.BASE_URL = "https://www.wikiart.org/en/"
        self.BASE_URL_API = self.BASE_URL + "api/2/"
        self.BASE_URL_MOVEMENT = self.BASE_URL + 'artists-by-art-movement/'
        self.ARTIST_DATA_URL = self.BASE_URL_API + "UpdatedArtists"

    def write_artist_data_into_json_file(self):
            artists = requests.get(ARTIST_DATA_URL).json()
            all_artists = artists['data']

            with open('artistdata.json', 'w') as outfile:
                while artists['hasMore']:
                    print('fetching next: pagination token',artists['paginationToken'])
                    url = BASE_URL + "?paginationToken=" + artists['paginationToken']
                    artists_next_page = requests.get(url).json()
                    next_artists = artists_next_page['data']
                    time.sleep(0.25)
                    all_artists = all_artists + next_artists
                    artists = artists_next_page
                json.dump(all_artists, outfile, indent=4)

from pprint import pprint

with open('artistdata.json', 'r', encoding='utf-8') as data_file:    
    data = json.load(data_file)
    pprint(data)
like image 890
zinyosrim Avatar asked May 17 '17 22:05

zinyosrim


People also ask

How do I read a JSON file in Python?

Reading From JSON Python has a built-in package called json, which can be used to work with JSON data. It's done by using the JSON module, which provides us with a lot of methods which among loads() and load() methods are gonna help us to read the JSON file.

What is JSON decode error in Python?

JSONDecodeError: Extra data" occurs when we try to parse multiple objects without wrapping them in an array. To solve the error, wrap the JSON objects in an array or declare a new property that points to an array value that contains the objects. Here is a very simple example of how the error occurs. main.py.

How do you convert a string to a JSON object in Python?

you can turn it into JSON in Python using the json. loads() function. The json. loads() function accepts as input a valid string and converts it to a Python dictionary.

Can I read JSON with pandas?

Reading JSON Files using PandasTo read the files, we use read_json() function and through it, we pass the path to the JSON file we want to read. Once we do that, it returns a “DataFrame”( A table of rows and columns) that stores data.


2 Answers

json.load() is for loading a file. json.loads() works with strings.

like image 64
zipa Avatar answered Oct 22 '22 01:10

zipa


3 ways to load a json file:

import json
import ast
with open(file_path) as file:
    data1 = json.load(file)
    data2 = json.loads(file.read())  
    data3 = ast.literal_eval(file.read())

You should use json.load whenever possible, but sometimes the JSON file is not strictly in the correct format (e.g. single quotes instead of double quotes). A solution is to use ast.literal_eval().

like image 26
victortv Avatar answered Oct 21 '22 23:10

victortv