Best way to modify a file when using pipes?

Question

I often have shell programming tasks where I run into this pattern:

cat file | some_script > file

This is unsafe - cat may not have read in the entire file before some_script starts writing to it. I don't really want to write the result to a temporary file (its slow, and I don't want the added complication of thinking up a unique new name).

Perhaps, there is there is a standard shell command that will buffer a whole stream until EOF is reached? Something like:

cat file | bufferUntilEOF | script > file

Ideas?

Answer 1

Like many others, I like to use temporary files. I use the shell process-id as part of the temporary name so that if multiple copies of the script are running at the same time, they won't conflict. Finally, I then only overwrite the original file if the script succeeds (using boolean operator short-circuiting - it's a little dense but very nice for simple command lines). Putting that all together, it would look like:

some_script < file > smscrpt.$$ && mv smscrpt.$$ file

This will leave the temporary file if the command fails. If you want to clean up on error, you can change that to:

some_script < file > smscrpt.$$ && mv smscrpt.$$ file || rm smscrpt.$$

BTW, I got rid of the poor use of cat and replaced it with input redirection.

Answer 2

Another option is just to read the file into a variable:

file_contents=$(cat file)
echo "$file_contents" | script1 | script2 > file