How to remove Task from collection after completed

Question

Let's say I have a collection of System.Threading.Tasks.Task:

HashSet<Task> myTasks = new HashSet<Task>();

...and I periodically feed more into the collection as I have more data that needs to be processed:

foreach (DataItem item in itemsToProcess)
    myTasks.Add(
        Task.Factory.StartNew(
            () => Process(item),
            cancellationToken,
            TaskCreationOptions.LongRunning,
            TaskScheduler.Default));    

Since Tasks remain in TaskStatus.RanToCompletion status after finishing instead of just disappearing, they would remain in the collection until explicitly removed and the collection would grow indefinitely. The Tasks need to be pruned to prevent this.

One approach I have looked at is giving the Task access to the collection and having it remove itself at the very end. But I am also looking at an architecture where I would have to remove a Task that my component has not created. My first thought is to attach a trigger or event to completion of each Task, something like this:

foreach (Task task in createdSomewhereElse)
{
    lock (myTasks) myTasks.Add(task);
    task.WhenTaskIsCompleted += 
        (o, ea) => { lock(myTasks) myTasks.Remove(task); };
    task.Start();
}

...but Task has no such event. Is there any good way to accomplish what I'm looking for? Something like this:

Answer 1

You certainly can attach a trigger for when a task is completed: Task.ContinueWith (and its generic equivalent). That would probably be good enough for you.

You may also wish to use ConcurrentDictionary as a sort of poor-man's concurrent set - that way you wouldn't have to lock when accessing the collection. Just use the Keys property when iterating, and use anything you like as the value.

Answer 2

Why do you need to keep the tasks in a collection?

Why not use a solution based on BlockingCollection and Parallel.ForEach

var sources = new BlockingCollection<DataItem>();

Task.Factory.StartNew(() => {
    Parallel.ForEach(sources.GetConsumingPartitioner(),
                     item => Process(item));
});

Now you can just feed your items into the blocking collection and they will automatically be processed.

foreach (DataItem item in itemsToProcess)
    sources.Add(item);

You can use sources.Countand foreach (DataItem item in sources) to see non-processed items. (A difference from your solution is that you can't see items currently processing)

Answer 3

Use the ContinueWith to set a action which removes the task from the set.