How to specify the directory that dask uses for temporary files?

Question

Dask seems to write to the /tmp folder. How can I change the folder that dask uses for temporary files?

Answer 1

Setting TMPDIR could potentially cause problems as it might also effect other applications. An alternative is to use dask.config.set

>>> import dask
>>> with dask.config.set({'temporary_directory': '/path/to/tmp'}):
...     pass

You could also add the lines

temporary_directory: /path/to/tmp

to .dask/config.yaml (in your home directory) configuration docs

Answer 2

For some reason the accepted code doesn't work for me. I get this error: AttributeError: 'set' object has no attribute 'items'

Here's a version that works: dask.config.set(temporary_directory='/path/to/tmp')

Also note that you have to do this before you create your Client.

Answer 3

Setting the TMPDIR environment variable to the desired location via export TMPDIR=/my/path seems to work.