Dask fails with freeze_support bug

Question

I try to run a very simple Dask program like the following:

# myfile.py
from dask.distributed import Client

client = Client()

But when I run this program, I get this odd error

    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

Answer 1

When calling Client() or LocalCluster() you are starting some new processes in your program. Python doesn't like it when modules or scripts start processes like this.

To resolve the problem, include your code in a if __name__ == "__main__": block like the following:

# client = Client()

if __name__ == '__main__':
    client = Client()
    ...

Alternatively you can choose to use threads rather than processes safely.

client = Client(processes=False)

Also, this problem will not occur if you run within an interactive session like IPython or Jupyter