Python Errno 9 Bad file descriptor in Mac OS X

Question

I have the following code running without any problem multiple times after each other in Linux:

def test_ftp(ftpserver):
    with FTP() as f:
        f.connect("localhost", port=ftpserver.server_port)
        f.login("M1", "bachmann")
        f.cwd("/")
        f.mkd("FOO")
        f.quit()

The same tests can only be run once in MacOS X, after that it will simply hang. Rebooting the machines, is the only way I can re-run the tests.

ftpserver is a test fixture defined in pytest-localftpserver, I am posting the code for this fixture here for the reason that I suspect that it is the cause of error:

class MPFTPServer(multiprocessing.Process):

    def __init__(self, username, password, ftp_home, ftp_port, **kwargs):
        self._server = SimpleFTPServer(username, password, ftp_home, ftp_port)
        self.server_home = self._server.ftp_home
        self.anon_root = self._server.anon_root
        self.server_port = self._server.ftp_port

        super().__init__(**kwargs)

    def run(self):
        self._server.serve_forever()

    def join(self):
        self._server.stop()

    def stop(self):
        self._server.stop()

@pytest.fixture(scope="session", autouse=True)
def ftpserver(request):
    """The returned ``ftpsever`` provides a threaded instance of
    ``pyftpdlib.servers.FTPServer`` running on localhost.
    ...
    """

    from pytest_localftpserver.plugin import MPFTPServer
    ftp_user = os.getenv("FTP_USER", "fakeusername")
    ftp_password = os.getenv("FTP_PASS", "qweqwe")
    ftp_home =  os.getenv("FTP_HOME", "")
    ftp_port = int(os.getenv("FTP_PORT", 0))
    server = MPFTPServer(ftp_user, ftp_password, ftp_home, ftp_port)
    # This is a must in order to clear used sockets
    server.daemon = True
    server.start()
    yield server
    server.join()

Can you tell why this code "works repeatedly" in Linux but not in MacOSX?

update

Digging a bit further, I found that the ftp server will not even start, hence the hanging. The code crashes with the following message:

    Process MPFTPServer-1:
    Traceback (most recent call last):
      File "/opt/pkg/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
        self.run()
      File "/Users/w/.virtualenvs/controller_config/lib/python3.5/site-packages/pytest_localftpserver/plugin.py", line 81, in run
        self._server.serve_forever()
      File "/Users/w/.virtualenvs/controller_config/lib/python3.5/site-packages/pyftpdlib/servers.py", line 207, in serve_forever
        self.ioloop.loop(timeout, blocking)
      File "/Users/w/.virtualenvs/controller_config/lib/python3.5/site-packages/pyftpdlib/ioloop.py", line 348, in loop
        poll(soonest_timeout)
      File "/Users/w/.virtualenvs/controller_config/lib/python3.5/site-packages/pyftpdlib/ioloop.py", line 709, in poll
        timeout)
    OSError: [Errno 9] Bad file descriptor

Answer 1

OK, apparently the bad file descriptor in Mac OS X is known:

that's what happens if you create an IOLoop before a fork and then try to use it in the child process. If you're going to use fork, you have to do it before anything creates the singleton IOLoop.

So the solution was simply to start the server instance inside the run method, instead in __init__:

class MPFTPServer(multiprocessing.Process):

    def __init__(self, username, password, ftp_home, ftp_port, **kwargs):
        self.username = username
        self.password = password
        self.server_home = ftp_home
        self.server_port = ftp_port

        super().__init__(**kwargs)

    def run(self):
        self._server = SimpleFTPServer(self.username, self.password,
                                       self.server_home, self.server_port)
        self._server.serve_forever()