Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Race condition while moving files on Linux

Suppose I have two scripts. The first one puts (with mv command) some files into a directory, the second one checks the directory once in a while and processes the files. The situation I'm concerned about is when the second script starts processing of the file which is only partly moved at the moment. Can this happen in real life on XFS file system?

like image 355
nab Avatar asked Feb 27 '12 10:02

nab


People also ask

What is race condition in Linux?

A race condition is an undesirable situation that occurs when a device or system attempts to perform two or more operations at the same time, but because of the nature of the device or system, the operations must be done in the proper sequence to be done correctly.

How do you avoid race condition in Unix?

To avoid race condition we need Mutual Exclusion. Mutual Exclusion is someway of making sure that if one process is using a shared variable or file, the other processes will be excluded from doing the same things.

How can race conditions be prevented?

To avoid race conditions, any operation on a shared resource – that is, on a resource that can be shared between threads – must be executed atomically. One way to achieve atomicity is by using critical sections — mutually exclusive parts of the program.


2 Answers

It depends on where you're moving the files from. mv WITHIN a single filesystem is atomic, otherwise it must do a copy which is not atomic (followed by a delete of the original file), and is prone to the kind of race condition you mention.

FWIW, this is normal POSIX semantics, nothing particular to XFS.

like image 168
janneb Avatar answered Sep 28 '22 02:09

janneb


Race condition would not occure in your case in XFS file system. However XFS allows multiple processes to read and write a file at once by using flexible locking scheme in contrast to Unix file systems single threaded inode lock. XFS tack care of serializing the writes on the same region by multiple processes .

XFS uses direct I/O for accessing the file.Direct I/O allows an application to specify that its data not to be cached in the buffer cache.

When using normal, buffered I/O, multiple readers can access the file concurrently, but only a single writer is allowed access to the file at a time. When using direct I/O, multiple readers and writers can access the file simultaneously.

like image 31
raj_gt1 Avatar answered Sep 28 '22 04:09

raj_gt1