Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Open JSON files in different directory - Python3, Windows, pathlib

I am trying to open JSON files located in a directory other than the current working directory (cwd). My setting: Python3.5 on Windows (using Anaconda).

from pathlib import *
import json

path = Path("C:/foo/bar")
filelist = []
for f in path.iterdir():
    filelist.append(f)

for file in filelist:
    with open(file.name) as data_file:    
        data = json.load(data_file)

In this setting I have these values:

file >> C:\foo\bar\0001.json
file.name >> 0001.json

However, I get the following error message:

---> 13     with open(file.name) as data_file:
     14         data = json.load(data_file)

FileNotFoundError: [Errno 2] No such file or directory: '0001.json'

Here is what I tried so far:

Use .joinpath() to add the directory to the file name in the open command:

with open(path.joinpath(file.name)) as data_file:
    data = json.load(data_file)

TypeError: invalid file: WindowsPath:('C:/foo/bar/0001.json')

Used .resolve() as that works for me to load CSV files into Pandas. Did not work here.

for file in filelist:
    j = Path(path, file.name).resolve()
    with open(j) as data_file:    
        data = json.load(data_file)

Since I'm on Windows write path as (and yes, the file is in that directory):

path = Path("C:\\foo\\bar") #resulted in the same FileNotFoundError above.

Instantiate path like this:

path = WindowsPath("C:/foo/bar")
#Same TypeError as above for both '\\' and '/'
like image 895
raummensch Avatar asked Dec 05 '22 17:12

raummensch


2 Answers

The accepted answer has a lot of redundants - re-collected generator and mixed with statement with pathlib.Path. pathlib.Path is awesome solution to handle paths especially if we want to create scripts which may work with Linux and Windows.

# modules
from pathlib import Path
import json

# static values
JSON_SUFFIXES = [".json", ".js", ".other_suffix"]

folder_path = Path("C:/users/user/documents")
for file_path in folder_path.iterdir():
    if file_path.suffix in JSON_SUFFIXES:
        data = json.loads(file_path.read_bytes())

Just adding modification for new users. pathlib.Path works with Python3.

like image 173
Marek Gancarz Avatar answered Jan 28 '23 18:01

Marek Gancarz


Complete solution; thanks @eryksun:

from pathlib import *
import json

path = Path("C:/foo/bar")
filelist = []
for f in path.iterdir():
    filelist.append(f)

for file in filelist:
    with open(str(file) as data_file:    
        data = json.load(data_file)

This line works as well:

with file.open() as data_file:
like image 27
raummensch Avatar answered Jan 28 '23 18:01

raummensch