How do you grow beyond web frameworks to create your own application framework? [closed]

Question

Web frameworks are great. I consider rolling-your-own without considering popular open source libraries is a design smell. So if someone were going to start a web project without using a popular server-side web framework like Rails and a popular client-side framework like jQuery, I'd think that they were either crazy, ignorant, or very niche.

That said, there are lots of things that web frameworks don't try to do for you. IMHO frameworks like Rails and jQuery have been successful because they try to take you 80% there, leaving the next 20% for you to do. Doing 80% allows them to be flexible enough to be widely used without becoming too constrictive or awkward. So the question becomes, what do you do with that 20% remaining, especially as your application grows larger?

We've developed and maintained a Rails/jQuery-UI application for the past 1.5 years. As stated, the unconstricted power of those frameworks proved great for getting us up to speed quickly, maintaining our productivity, and reinforcing good design. However, over the past few months, I've started to think that we should be able to develop and deploy new features even faster, and I've started to feel that we haven't build enough atop the rudiments that Rails and jQuery gives us. New features seemingly have to developed from that 80% point every time, instead of a preferable 90-95% point.

Why are your strategies for growing beyond web frameworks? What techniques or technologies that you've used to move that 80% starting point closer to 90-95%? What specific hurdles to you encounter or overcome building your own application framework or toolkit? What were the rubs of developing on vanilla Rails and jQuery that pushed you to look for tighter application integration?

Answer 1

Frameworks and libraries leave that "20%" you note so that you can build on top of them. If you are finding that you are still working, barebones, at that 80% level every time you need to add a new feature or functionality then you haven't done anything.

Personally, I've used many PHP frameworks where I build custom libraries and functionality on top that helps take my projects to that 90-95% level. That difference of 15% of your project is very important. A few examples of that code are things like: utility functions, permission systems, internal apis and template managers (which help render data with your views).

As for client side, Javascript, libraries (jQuery, Prototype, Dojo etc.) it sounds like you haven't thought long term. More and more people are realizing they need to start with a strictly Javascript application structure before thinking about which library to use. Libraries provide some standard ways to bind events, select elements, etc. but none seem to really have large scale application logic built in. You need to build that yourself.

Loose coupling (or Pub/Sub - Publish Subscribe) has become really popular and there are some great libraries that help with MVC and view state like jQuery BBQ and Backbone.js (like @Raynos suggested). This logic helps you scale and properly manage new functionality in a way that is standard across your app. That said, you should still understand and begin with a purely library-less application structure you understand. I've written a good 101 post about this here (http://darcyclarke.me/development/javascript-applications-101/) and Addy Osmani also gives great resources for this here (http://addyosmani.com/blog/large-scale-jquery/). A bit different then server-side, I suggest that you build that 15-20% before you dive into the decision of which library to use. They do, after all, have many of the same features and shouldn't be relied on solely to build your client side app.

I still think you're better off having these tools in place, rather then building your own from scratch, but I think you need to start building your own set of tools on top of them.

Answer 2

I don't really do much with server-side frameworks because our ASP.NET backend already handles the 90% and all the custom server-side controls everyone else has written deals with that last 5%.

In term's of client side there is little you can do apart from writing generic re-useable controls. The main reason I use jQuery is because it abstracts cross-browser compliance away. I use it just like I would with JavaScript except it works effortlessly in IE.

Build re-useable controls on top of jQuery. Make custom plugin's. Make all the custom code you've written far more generic so you can re-use it from project to project.

I recommend you take a look at backbone.js. It's a client-side MVC framework that really allows you to customize your web applications. Building on top of such a MVC framework makes the code very easy to extend and very manageable. The nice thing about this is that you have a lot of control and you can set up your own generic framework on top of that allows for re-use, re-use & re-use.

One of the important things to remember is to delegate cross-browser compliance to an underlying library like jQuery and then build abstractions on top of it.

In my personal experience the generic bad code lying around everywhere is dragging us down a lot more then the limitations of jQuery. Maybe if everyone wrote great code we would notice limitations of jQuery. I can't really see any limitations of the ASP.NET framework yet.