What would you recommend for a high traffic ajax intensive website?

Question

For a website like reddit with lots of up/down votes and lots of comments per topic what should I go with?

Lighttpd/Php or Lighttpd/CherryPy/Genshi/SQLAlchemy?

and for database what would scale better / be fastest MySQL ( 4.1 or 5 ? ) or PostgreSQL?

Answer 1

I can't speak to the MySQL/PostgreSQL question as I have limited experience with Postgres, but my Masters research project was about high-performance websites with CherryPy, and I don't think you'll be disappointed if you use CherryPy for your site. It can easily scale to thousands of simultaneous users on commodity hardware.

Of course, the same could be said for PHP, and I don't know of any reasonable benchmarks comparing PHP and CherryPy performance. But if you were wondering whether CherryPy can handle a high-traffic site with a huge number of requests per second, the answer is definitely yes.

Answer 2

The ideal setup would be close to this:

In short, nginx is a fast and light webserver/front-proxy with a unique module that let's it fetch data directly from memcached's RAM store, without hitting the disk, or any dynamic webapp. Of course, if the request's URL wasn't already cached (or if it has expired), the request proceeds to the webapp as usual. The genius part is that when the webapp has generated the response, a copy of it goes to memcached, ready to be reused.

All this is perfectly applicable not only to webpages, but to AJAX query/responses.

in the article the 'back' servers are http, and specifically talk about mongrel. It would be even better if the back were FastCGI and other (faster?) framework; but it's a lot less critical, since the nginx/memcached team absorb the biggest part of the load.

note that if your url scheme for the AJAX traffic is well designed (REST is best, IMHO), you can put most of the DB right in memcached, and any POST (which WILL pass to the app) can preemptively update the cache.

Answer 3

On the DB question, I'd say PostgreSQL scales better and has better data integrity than MySQL. For a small site MySQL might be faster, but from what I've heard it slows significantly as the size of the database grows. (Note: I've never used MySQL for a large database, so you should probably get a second opinion about its scalability.) But PostgreSQL definitely scales well, and would be a good choice for a high traffic site.