Elegantly Coding a try: except chain

Question

I have a in concurrent.futures that usually works with chunksize=1. However, occasionally I hit a large dataset that requires a larger chunksize. Currently I have solved this issue with the following code:

for i in datasets:
    try:
        with concurrent.futures.ProcessPoolExecutor() as executor:
            results=tuple(executor.map(do_something, parameters, chunksize=1)
    except concurrent.futures.process.BrokenProcessPool:
        try:
            with concurrent.futures.ProcessPoolExecutor() as executor:
                results=tuple(executor.map(do_something, parameters, chunksize=2)
        except concurrent.futures.process.BrokenProcessPool:
            try:
                with concurrent.futures.ProcessPoolExecutor() as executor:
                    results=tuple(executor.map(do_something, parameters, chunksize=4)
            etc. etc. etc....
            except concurrent.futures.process.BrokenProcessPool:
                   print('code failed')

This works fine, but is obviously really inelegant and ugly. Anyway that I can do this more simply?

Answer 1

You can use a for loop to iterate through the chunk sizes, break the loop if it succeeds, or enter an else block to output the error message:

for i in datasets:
    for chunksize in 1, 2, 4:
        try:
            with concurrent.futures.ProcessPoolExecutor() as executor:
                results=tuple(executor.map(do_something, parameters, chunksize=chunksize)
            break
        except concurrent.futures.process.BrokenProcessPool:
            pass
    else:
        print('code failed')