How do you use Tasks to spin off lots of "Fire and Forget" work?

Question

I have a Windows application that I want to introduce TPL functionality to. The scenario is that I have a class with a Process method that has a collection of MailMessages passed to it, as well as the current IMAP connection (using AE.Net.Mail).

I want to spin off as many threads as possible to an Execute method in another class that takes a single MailMessage item, and the MailMessage to the DB, and then uses the IMAP connection to delete the MailMessage from the server.

I'm not worried too much about keeping track of the processes - I'm dealing with large numbers of emails, and not worried if I get some errors in writing to the DB or deleting. I just need the application to get through a large number of MailMessages as quick as possible.

I have been playing around with Task<MailMessage>.Factory.StartNew but I dont really know what I am doing. I don't seem to be able to kick it off...here's what I have tried:

Task test = Task.Factory.StartNew(() =>
{
  foreach (var mailMessage in _mms)
  {
    new ProcessMessage().Execute(mailMessage, imapConn);
  }

});

I'm pretty sure I should not have a loop in the lamda expression, when I run this it does not seem to go into ProcessMessage.Execute.

Answer 1

You definitely shouldn't have a loop in your lamdba expression. Try this:

_mms.ForEach(mms => {
    Task.Factory.StartNew(() => ProcessMessage().Execute(mailMessage, imapConn))
});

If you're not worried about keeping track of results or anything you don't need to save an instance of a Task such as Task test = .... you can just kickstart the method execution in a new thread using Task.Factory.StartNew() This way we can simply start up a new Task for each mail message you want to process and let the thread pool take care of things for us.

Also, Task<MailMessage>.Factory.StartNew would be used to set up a method call in another thread that returns a MailMessage so if you are calling a void method you don't need to do this. The Task<object> syntax always refers to the return type of the method you are starting up with a new Task.

Answer 2

Right now you are executing your foreach loop as a separate task, but you probably want to execute each iteration as a separate task. You should try the Parallel.ForEach:

        Parallel.ForEach(_mms, mailMessage =>
            {
                new ProcessMessage().Execute(mailMessage, imapConn);
            });

This will execute iterations in parallel, which seems to be what you are trying to do.

Answer 3

Another option is to use .AsParallel() and .ForAll() on the collection:

_mms.AsParallel()
    .ForAll(mm => ProcessMessage().Execute(mm, imapConn));

Answer 4

That should do each execution in separate thread (so you know what you are doing now :) )

  foreach (var mailMessage in _mms)
  {
     ThreadPool.QueueUserWorkItem(delegate
     {
        new ProcessMessage().Execute(mailMessage, imapConn);
     });
  }

or

  foreach (var mailMessage in _mms)
  {
      new Thread(delegate() { new ProcessMessage().Execute(mailMessage, imapConn); }).Start();
  }