Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python append multiple files in given order to one big file

Tags:

I have up to 8 seperate Python processes creating temp files in a shared folder. Then I'd like the controlling process to append all the temp files in a certain order into one big file. What's the quickest way of doing this at an os agnostic shell level?

like image 406
Martlark Avatar asked Apr 01 '11 06:04

Martlark


People also ask

How do you merge files in a folder in Python?

To merge all excel files in a folder, use the Glob module and the append() method. Note − You may need to install openpyxl and xlrd packages.

Can you write to two files at once in Python?

Python provides the ability to open as well as work with multiple files at the same time. Different files can be opened in different modes, to simulate simultaneous writing or reading from these files. An arbitrary number of files can be opened with the open() method supported in Python 2.7 version or greater.


1 Answers

Just using simple file IO:

# tempfiles is a list of file handles to your temp files. Order them however you like f = open("bigfile.txt", "w") for tempfile in tempfiles:     f.write(tempfile.read()) 

That's about as OS agnostic as it gets. It's also fairly simple, and the performance ought to be about as good as using anything else.

like image 77
Rafe Kettler Avatar answered Sep 25 '22 14:09

Rafe Kettler