Parallel writing to a file from multiple processes by using echo

Question

I have written a function in our ERP-System which writes a log to a file on the server by simply "echoing" it to the logfile.

echo "SOME LOGDATA" >> /users/erp/log/LOGMSG

Every time a user triggers a specific event, the LOG function is called.

What would happen if 2 users trigger the LOG event exactly at the same time?

Does "echo" take care of file locking? In my opinion it has to be the linux kernel or the bash who has to takes care, that a file is not written by 2 command line instructions simultaneously.

I wrote a testcase to force this condition (some 1000 LOG calls within one second) and it seems that my thoughts are right but I can not be really sure, that these calls are executed at the same time on the bash.

Answer 1

As explained here, echo is guaranteed to be atomic only when writing sequences shorter than the smaller of PIPE_BUF and the size of the stdout buffer (which is most likely BUFSIZ).

For longer sequences, you need to locking. Use can use lockfile-create and lockfile-check.

Answer 2

You can use GNU Parallel as a mutex/semaphore ensuring that only one echo runs at a time like this:

sem --id mysem echo "System crashed"  >> log.txt

GNU Parallel is installed on most Linux distros, and is readily available for OSX via homebrew.

You can test the idea by starting a loop in each of two, or more, terminals and having each one do this:

for i in {1..100}; do sem --id mysem echo hi >> log ; done

and at the end you will see there are exactly 100 lines per terminal in your log.

Answer 3

Be careful not to reinvent the wheel, in *nix systems there is syslog to handle logging in a proper way. The problem with echo to a file is that you can enforce mutual exclusion but the hardest part would be to handle serialization, the fact that one echo writes before another does not mean that the logged operation actually executed first (nor in parallel).