Generating PDF for 90K records

Question

Currently I am using LocalReport. Render to create PDF's for 90K records. Using normal 'for' loop, it takes around 4 hours to create PDF only. I have tried many options.

1. Tried with Parallel. Foreach with and without setting MaxDegreeOfParallelism with different values. There are 2 processors in my system. With setting MaxDegreeOfParallelism(MDP) =4, it is taking the time as normal 'for' loop. I thought increasing MDP to 40 will speed up the process. But didn't get expected results since it took 900 minutes.

2. Used

   var list=List<Thread ()>;
   foreach (var record in records) {
       var thread = new Thread (=> GeneratePDF());
       thread.Start();
       list.Add(thread);
   }
   foreach(var listThreads in thread){
       listThreads. Join();
   

   }

I used the code above like that. But it ended up creating too many threads and took so longer time.

I need help in using Parallel. Foreach to speed up the process of creating PDF's for 90K records. Suggestions to change the code is also acceptable. Any help would be much appreciated.

Thanks

Answer 1

I don't know any pdf generators, so I can only assume there is a lot overhead in initializing and in finalizing things. That's what I'd do:

Find an open source pdf generator.

Let it generate a few separate pieces of a pdf - header, footer, etc.

Dig the code to find where the header/footer is done and try work around them to reuse generator states without running through the entire process.

Try to stich together a pdf from stored states and a generator writing only the different parts.