Does Dask support functions with multiple outputs in Custom Graphs?

Question

The Custom Graphs API of Dask seems to support only functions returning one output key/value.

For example, the following dependency could not be easily represented as a Dask graph:

    B -> D
   /      \
A-         -> F
   \      /
    C -> E

This can be worked around by storing a tuple under a "composite" key (e.g. "B_C" in this case) and then splitting it by getitem() or similar. However, that can lead to inefficient execution (e.g. unnecessary serialization) and reduce the clarity of DAG visualizations.

Is there a better way or is this currently not supported?

Answer 1

Short answer

No, but it shouldn't matter.

Programming interface

You are correct that the correct way to manage multiple outputs with Dask is to use getitem. In terms of programming interface, the standard way to do this with dask.delayed is with getitem as you suggest. Here is an example:

from dask import delayed

@delayed(pure=True)
def minmax(a, b):
    if a > b:
        return a, b
    else:
        return b, a

result = minmax(1, 2)
min, max = result[0], result[1]

Performance

You raise an interesting question about performance. In practice using the distributed scheduler (which works just fine on a single machine) should handle this sort of situation just fine without performance penalty. The same would be true for the single-machine threaded scheduler.