I have a set of fairly large files (about 50 megabytes each, and at least a hundred of them), but I need to insert a small header (about 2 dozen lines) onto each one for processing purposes. I was hoping to write a script in either bash or python to do it, but I can't find a constant-time function that will let me insert at the front of a text file. If it is not constant time, I think it'll take too long to complete. Does anyone have experience with this issue?
Similar to Uwe's answer, but if your processing tool can only accept parameters as filenames, you can fake one up with mkfifo(1).
For example, in bash...
echo 'My header' > header.txt
echo 'My content' > content.txt
mkfifo fakefile.txt
cat header.txt content.txt > fakefile.txt &
cat fakefile.txt
...would stream the contents of the two files, rather than creating a new file.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With