How non blocking read/write throught remote FileSystem

Question

Is there a way to write and read files on a remote filesystem (such as NFS, SSHFS, or sambafs) in a way that read or write or even open return immediately with an error code? In fact I'm using Twisted and I want to know whether there is a safe way to access remote files without blocking my reactor.

Answer 1

In Twisted, for remote filesystems just like for any other blocking calls, you can use threads.deferToThread -- a reasonably elegant way to deal with pesky blocking syscalls!-)

Answer 2

This is actually very similar to my question asked here. It seems that the only way to get around the limitations of the operating system, at present, is to use threads or external processes to handle your file IO for you.

In a previous life (non-python or twisted, but very asynchronous), we ended up abstracting file IO out into a separate daemon that was essentially our 'file system worker'.

2.6.x versions of linux seem to have added more support for asychronous IO at the kernel level, with libaio being the support for it, but it looks pretty arcane and rather dubious in what it actually supports.