Concatenate multiple files into a single file object without creating a new file

Question

This question is related to Python concatenate text files

I have a list of file_names, like ['file1.txt', 'file2.txt', ...].

I would like to open all the files into a single file object that I can read through line by line, but I don't want to create a new file in the process. Is that possible?

with open(file_names, 'r') as file_obj:
   line = file_obj.readline()
   while line:
       ...

Answer 1

Use input from fileinput module. It reads from multiple files but makes it look like the strings are coming from a single file. (Lazy line iteration).

import fileinput

files= ['F:/files/a.txt','F:/files/c.txt','F:/files/c.txt']

allfiles = fileinput.input(files)

for line in allfiles: # this will iterate over lines in all the files
    print(line)

# or read lines like this: allfiles.readline()

If you need all the text in one place use StringIO

import io

files= ['F:/files/a.txt','F:/files/c.txt','F:/files/c.txt']


lines = io.StringIO()   #file like object to store all lines

for file_dir in files:
    with open(file_dir, 'r') as file:
        lines.write(file.read())
        lines.write('\n')

lines.seek(0)        # now you can treat this like a file like object
print(lines.read())

Answer 2

try something along this lines:

def read_files(*filenames):
    for filename in filenames:
        with open(filename,'r') as file_obj:
            for line in file_obj:
                yield line

you can call it with

for line in read_files("f1.txt", "f2.txt", "f3.txt"):
    #... do whatever with the line

or

filenames = ["f1.txt", "f2.txt", "f3.txt"]
for line in read_files(*filenames):
    #... do whatever with the line