Combine csv files with same name from different subfolders in to one csv

Question

I have three CSV files each for a particular filename for multiple files. Let's say there are a total 20 filenames so total 20* 3csv files in three different folders.

Folder A- 1001.CSV,1002.CSV,1003.CSV...
Folder B-1001.CSV,1002.CSV,1003.CSV
Folder C-1001.csv,1002.csv,1003.csv......

I want to get a single CSV file for each 1001,1002,1003,1004..... So total 20csv files

How can I do this? Since the files are in different folders glob is not working(or I don't know how to)

Answer 1

I made the following assumptions:

• all the subfolders will be rooted at some known directory "parentdir"
• each subfolder contains only relevant csv files
• the csv files do not contain any header/footer lines
• each record in the csv files is separated by a newline
• all of the records in each file are relevant

This should produce a "concat.csv" file in each subfolder with the contents of all the other files in that same folder. I used a snippet of code from this other answer on stackoverflow for actually concatenating the files.

import os
import fileinput

rootdir = 'C:\\Users\\myname\\Desktop\\parentdir'
os.chdir(rootdir)
children = os.listdir()
for i in children:
    path = os.path.join(rootdir, i)
    os.chdir(path)
    filenames = os.listdir()
    with open('concat.csv', 'w') as fout, fileinput.input(filenames) as fin:
        for line in fin:
            fout.write(line + '\n')