Will using multiple threads speed up my HTML file processing application?

Question

I just finished up my most complex and feature-laden WinForms application to date. It loads a list any number of HTML files, then loads the content of one, uses some RegEx to match some tags and remove or replace them (yes, yes, I've seen this. It works just fine, thanks Cthulu), then writes it to disk.

However, I noticed that ~200 files takes roughly 30 seconds to process, and after the first 5-10 seconds the program is reported as "Not Responding". I'm assuming it's not wise to do something like this guy did, as the hard drive is a bottleneck.

Perhaps it'd be possible to load as many as possible into memory, then process each one with a thread, write those, then load some more into memory?

At the very least, would creating a worker thread separate from the UI thread prevent the "Not Responding" issue? (This MSDN article covers what I was considering.)

I guess I'm asking if multithreading will offer any sort of speed improvement, and if so, what would be the best way of going about it?

Any help or advice is much appreciated!

Answer 1

Yes, you should start by using a Backgroundworker to decouple your work from the GUI. Handling a GUI event should never take too much time. Aim for 20ms, not 20s.

Then as a bonus you could see if the processing (CPU intensive part) can be split into independent jobs and execute them as TPL Tasks.

There is insufficient information to say if or how you should do that.