Run a task in a different threads in a console application

Question

I have done quite a bit of research on this but I still can't seem to get it right. I have to generate a pdf file with 1000 pages (I use a library) and need to do this N times for different data. The data is independent of each other and I can totally generate this in parallel, which is what i'm trying to do. Ideally I would like to have this done in say 10 threads , each thread generating the pdf in memory and saving it at the end. Say it takes 15 minutes per pdf (with 1000 pages), if I do this sequentially, that would be 150 minutes for 10 pdf files vs say 30 minutes if I use 10 threads. I know people are not very fond of threading, but how can I speed it up otherwise?

I was looking at ThreadPool but then I see this new Task in 4.0. I read that I can force each task to run in a separate thread if I use TaskCreationOptions.LongRunning but that does not seem to work for me. I tried to use ThreadPool too but since each PDF is generated from a url and for some reason, the WebRequest.Create(url) method seems to not execute when called from a threadpool? But I guess I would rather make the new Task library work.

This is what I have now but it still seems to execute sequentially.

Task myTask= Task.Factory.StartNew(() =>
                {
                  //code for the task.
                  //get html content
                  //generate pdf file.
                }
                }, new CancellationToken(false), TaskCreationOptions.LongRunning, TaskScheduler.Default);

myTask.Wait();

What am I doing wrong here? if you have any suggestions, please let me know. I can't go above .net 4.0 at the moment.

Answer 1

myTask.Wait() makes your control thread's execution halt until the task completes... You don't want to halt execution right after firing off one of these tasks.

What you want to do is create multiple tasks at once, start them, and then call Task.WaitAll(array) to wait for them ALL to complete instead of waiting for one at a time.

// Define your tasks and start them all
var task1 = Task.Factory.StartNew(() => { /*do something*/ });
var task2 = Task.Factory.StartNew(() => { /*do something*/ });
var task3 = Task.Factory.StartNew(() => { /*do something*/ });

// Wait for ALL tasks to finish
// Control will block here until all 3 finish in parallel
Task.WaitAll(new[] { task1, task2, task3 });