Scaling from one server to multiple servers

Question

Lets assume I'm running two servers: one for the website and the other for the DB.
Obviously, there will be a point when a bottleneck would occur for both servers.

• What should one need to do at this point? are there any best practices?
• What are the advantages of using cloud cloud services like AppEngine, Appfog etc?

I'm newbie at this zone so please be patient :)

Answer 1

There are 2 tiers to your application, the App server tier (for the Web site) and the database tier for the DB. Scaling each of these has much different consequences. In general scaling the app server tier is easy, especially if you do not store state (i.e., each request is independent). You can use a load balancer in front of your app server tier, and add/remove app servers as you need.

Scaling the database tier is a much bigger discussion and something you need to evaluate. It depends on the type of database you are using (e.g., MySQL or a NoSQL/Document engine like MongoDB). The newer NoSQL/Document engines provide scaling, and that works well for many applications, if you only need to access one thing at a time independently, like one object or one document. If you need relational data with guaranteed transactions, let's say because you have a shopping cart, then it can be more challenging to use those tools and a relational database with transactions is a more typical choice.

To scale a relational database, the first step is usually to load balance with a read-only slave, this is a duplicate (via replication) of your master server, and you can direct heavier reads to the slave. The slave is not generally guaranteed to be always current, so you need to send reads to the slave where it is fine if they are a little behind (like long running report type queries, or queries for data that doesn't change much).

After that a full scaling solution is required, and there are many ways to go about this. Some are "roll your own" solutions and others are commercial. Typically this is needed when you have very heavy write loads and can no longer get by with a bigger database server, or you need to distribute a lot of data (usually 100s of GBs to many TBs), and you need fast queries against that data. This is a big subject, but you can look at various solutions. I will say that we are such a vendor so I have a bias there :), but really you should learn much more about this if you need to go this way as you grow.

The best advice is to do the simpler things first, and plan for a longer term scalability strategy for the database tier that meets your needs.

Answer 2

If you are thinking about a service in which you can easily scale up/down your web server/database, I recommend you to look at a PaaS where you can easily get this approach paying every time for the services you use. Otherwise, you have to implement the mechanism through a load balancer where you can create multiple instances of your app, but then you have to manage difficult things like sticky sessions.

For databases, ClearDB (MySQL), MongoHQ (Mongo) or ElephantSQL (PostgreSQL) could easily do this work. They permit you to scale up/down your database. Feel free to look for more providers as these are the ones that I know. You will find for sure more on Internet.

Regarding your web application, a PaaS could provide you the benefit of scaling your application vertically or horizontally without taking care of the load balancer, the sticky session or the escalation process. Some PaaS also allow you to auto-scale your application depending on some parameters like the number of request you have. See here and also here an example of this.

Please note, that you can also implement all of this by yourself on an IaaS or through your own infrastructure, but in this case we will need more information about what you are exactly looking for. I am just giving you a way in with you can use what you need as a service without implementing it.