Best practice - load a lot of stuff in the application_start? [closed]

Question

I have a webshop with a lot of products and other content. Currently I load all content in to a global list at Application_Start, which takes aprox 15-25 seconds.

This makes the the site really fast, as I can get any product/content in O(1) time.

However, is this best practice?

Currently I got a webhotel which is not a VPS / Dedicated server, so it recycles the application from time to time, which gives random visitors load times up to 15-25 seconds (only to become a bigger number with more content). This is of course totally unacceptable, but I guess it would be solved with a VPS.

What is the normal way of doing this? I guess a webshop like Amazon probably don't load all their products into a huge list :-D

Any thoughts and ideas would be highly appreciated.

Answer 1

It looks like you've answered your question for your case "This is of course totally unacceptable".

If your goal O(1) normal request to database for single product is likely O(1) unless you need to have complicated joins between products. Consider trying to drop all your pre-caching logic and see if you have problem with performance. You can limit startup impact by lazy caching instead.

Large sites often use distributed caching like MemcaheD.

Answer 2

A more scalable setup is to set up a web service to provide the content, which the website calls when it needs it. The web service will need to cache frequently needed content to achieve fast response times.