Scale now or later?

Question

I am looking to start developing a relatively simple web application that will pull data from various sources and normalizing it. A user can also enter the data directly into the site. I anticipate hitting scale, if successful. Is it worth putting in the time now to use scalable or distributed technologies or just start with a LAMP stack? Framework or not? Any thoughts, suggestions, or comments would help.

Disregard my vague description of the idea, I'd love to share once I get further along.

Answer 1

Later. I can't remember who said it (might have been SO's Jeff Atwood) but it rings true: your first problem is getting other people to care about your work. Worry about scale when they do.

Definitely go with a well structured framework for your own sanity though. Even if it doesn't end up with thousands of users, you'll want to add features as time goes on. Maintaining an expanding codebase without good structure quickly becomes fairly horrible (been there, done that, lost the client).

btw, if you're tempted to write your own framework, be aware that it is a lot of work. My company has an in-house one we're quite proud of, but it's taken 3-4 years to mature.

Answer 2

Is it worth putting in the time now to use scalable or distributed technologies or just start with a LAMP stack?

A LAMP stack is scalable. Apache provides many, many alternatives.

Framework or not?

Always use the highest-powered framework you can find. Write as little code as possible. Get something in front of people as soon as you can.

Focus on what's important: Get something to work.

If you don't have something that works, scalability doesn't matter, does it?

Then read up on optimization. http://c2.com/cgi/wiki?RulesOfOptimization is very helpful.

Rule 1. Don't.

Rule 2. Don't yet.

Rule 3. Profile before Optimizing.

Until you have a working application, you don't know what -- specific -- thing limits your scalability.

Don't assume. Measure.

That means build something that people actually use. Scale comes later.