ASP.NET: Strategy for handling really large reports

Question

Maybe not specific to reports but still...

In my asp.net mvc web application there's a section for reports that show 5 columns of data that map almost directly to a table in the db.

The problem is, in some cases, the length of that report may exceed 40,000 records (I know, nobody can really process 40,000 records of data but the report is what it is) and as you can expect, it times out and throws an error.

The question is, what's a good way to process and deliver a report of that size? I thought about creating a small little console app that would build the report outside of the webserver but I'm kind of at a loss as to what direction to look into?

Answer 1

Does the report need to have up-to-the-minute data? If not, you can look at generating the report as a PDF at night (or whenever your server isn't busy) and just providing a link to the PDF. A scheduled task that runs a console app as you suggested could create the report and output it to a file. A lot of reporting tools like Crystal Reports will allow you to export the report to a PDF or an Excel spreadsheet. For that matter, you could generate the report on a completely different machine and then copy it over to the web server. This could allow you to update the report every hour (or whatever) without putting such a load on your web server.