Flaky file deletion under Windows 7?

Question

I have a Python test suite that creates and deletes many temporary files. Under Windows 7, the shutil.rmtree operations sometimes fail (<1% of the time). The failure is apparently random, not always on the same files, not always in the same way, but it's always on rmtree operations. It seems to be some kind of timing issue. It is also reminiscent of Windows 7's increased vigilance about permissions and administrator rights, but there are no permission issues here (since the code had just created the files), and there are no administrator rights in the mix.

It also looks like a timing issue between two threads or processes, but there is no concurrency here either.

Two examples of (partial) stack traces:

  File "C:\ned\coverage\trunk\test\test_farm.py", line 298, in clean
    shutil.rmtree(cleandir)
  File "c:\python23\lib\shutil.py", line 142, in rmtree
    raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)
WindowsError: [Errno 5] Access is denied removing xml_1

  File "C:\ned\coverage\trunk\test\test_farm.py", line 298, in clean
    shutil.rmtree(cleandir)
  File "c:\python23\lib\shutil.py", line 142, in rmtree
    raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)
WindowsError: [Errno 3] The system cannot find the path specified removing out

On Windows XP, it never failed. On Windows 7, it fails like this, across a few different Python versions (2.3-2.6, not sure about 3.1).

Anyone seen anything like this and have a solution? The code itself is on bitbucket for the truly industrious.

Answer 1

It's a long shot, but are you running anything that scans directories in the background? I'm thinking antivirus/backup (maybe Windows 7 has something like that built in? I don't know). I have experienced occasional glitches when deleting/moving files from the TSVNCache.exe process that TortoiseSVN starts -- seems it watches directories for changes, then presumably opens them for scanning the files.