What is the lifetime of the arguments of std::async?

Question

It appears that arguments of a function executed via std::async share the lifetime of the future:

#include <iostream>
#include <future>
#include <thread>

struct S
{
    S() {
        std::cout << "S() " << (uintptr_t)this << std::endl;
    }

    S(S&& s) {
        std::cout << "S(&&) " << (uintptr_t)this << std::endl;
    }

    S(const S& s) = delete;

    ~S() {
        std::cout << "~S() " << (uintptr_t)this << std::endl;
    }
};

int main()
{
    {
        std::cout << "enter scope" << std::endl;
        auto func = [](S&& s) {
            std::cout << "func " << (uintptr_t)&s << std::endl;
            auto x = S();
        };
        S s;
        auto fut = std::async(std::launch::async, func, std::move(s));
        std::cout << "wait" << std::endl;
        std::this_thread::sleep_for(std::chrono::seconds(5));
        fut.get();
        std::cout << "exit scope" << std::endl;
    }
    return 0;
}

Results in:

    enter scope
  ++S() 138054661364        main's variable
  | S(&&) 138054661108 ++   std::async's internal copy
+--+S(&&) 138054659668  |   std::async's internal copy
| | S(&&) 138054922824 +--+ func's argument
+--+~S() 138054659668   | |
  | ~S() 138054661108  ++ |
  | func 138054922824     |
  | S() 138057733700   +  |  local variable
  | ~S() 138057733700  +  |
  | wait                  |
  | exit scope            |
  | ~S() 138054922824  +--+
  ++~S() 138054661364

It looks like the underlying implementation (MSVS 2015 U3) creates the final version of the argument at the address 138054922824, but does not destroy it until future is destroyed.

It feels like this breaks the RAII promise as the function implementation may relay on destructors of the arguments being called upon exit.

Is this a bug or the exact lifetime of the arguments passed to std::async is unknown? What does the standard say about this?

Answer 1

Following up on my previous comment with an actual answer…

I have encountered the same behavior with libstdc++. I did not expect this behavior, and it resulted in a deadlock bug in my code (thankfully, due to a wait timeout, this only caused a delay in program termination). In this case, it was the task object (by which I mean the function object f) that was not destroyed after the task finished execution, only on destruction of the future, however, it is likely that the task object and any arguments are treated in the same manner by the implementation.

The behavior of std::async is standardized in [futures.async].

(3.1) If launch​::​async is set in policy, calls INVOKE(DECAY_­COPY(std​::​forward<F>(f)), DECAY_­COPY(std​::​forward<Args>(args))...) ([func.require], [thread.thread.constr]) as if in a new thread of execution represented by a thread object with the calls to DECAY_­COPY() being evaluated in the thread that called async. Any return value is stored as the result in the shared state. Any exception propagated from the execution of INVOKE(DECAY_­COPY(std​::​forward<F>(f)), DECAY_­COPY(std​::​forward<Args>(args))...) is stored as the exceptional result in the shared state. The thread object is stored in the shared state and affects the behavior of any asynchronous return objects that reference that state.

The wording, by using DECAY_COPY without naming the results and inside an INVOKE expression, does strongly suggest the use of temporary objects that are destroyed at the end of the full expression containing the INVOKE, which happens on the new thread of execution. However, this is not enough to conclude that the (copies of the) arguments, do not outlive the function call by more than the processing time it takes to clean them up (or any "reasonable delay"). The reasoning for it goes like this: Basically the standard requires that the objects are destroyed when the thread of execution completes. However, the standard does not require that the thread of execution completes before a waiting call is made or the future is destroyed:

If the implementation chooses the launch​::​async policy,

(5.3) a call to a waiting function on an asynchronous return object that shares the shared state created by this async call shall block until the associated thread has completed, as if joined, or else time out ([thread.thread.member]);

So, the waiting call could cause the thread to complete and only then wait on its completion. Under the as-if rule, the code could actually do worse things if they only appear to have this behavior, such as blatantly storing the task and/or arguments in the shared state (with the caveat to immediately follow). This does appear to be a loophole, IMO.

The behavior of libstdc++ is such that even an unconditional wait() is not enough to cause task and arguments to be destroyed – only a get() or destruction of the future will. If share() is called, only destruction of all copies of the shared_future is sufficient to cause the destruction. This appears to be a bug indeed, as wait() is certainly covered by the term "waiting function" in (5.3), and cannot time out. Other than that, the behavior seems to be unspecified – whether that's an oversight or not.

My guess as to why implementations seem to put the objects in the shared state is that this is much easier to implement than what the standard would literally suggest (making temporary copies on the target thread, synchronous with the call of std::async).

It seems like an LWG issue should be brought up about this. Unfortunately, any fix for this is likely to break the ABI of multiple implementations, and it may therefore take years until the behavior is reliably fixed in deployments, even if the change is approved.

Personally, I have come to the unfortunate conclusion that std::async has so many design and implementation issues that it is next to useless in a non-trivial application. The aforementioned bug in my code has been resolved by me replacing the offending use of std::async by uses of my own (dependency tracking) thread pool class, which destroys the task including all captured objects ASAP after the task finishes execution. (It simply pops the task info object, which contains the type-erased task, the promise and so on, from the queue.)

UPDATE: It should be noted that libstdc++'s std::packaged_task has the same behavior, the task appears to be moved into the shared state and will not be destroyed when the std::packaged_task is, as long as get() or any future destructors are pending.