The Web 2.0 Ecosystem/Stack

Question

Being new to front-end website development, I can understand some stuff, things like routes, ORM, etc. What I don't understand is how they all play together. My understanding is, there are a bunch of components for a website built with Pyramid/Django etc:

1. A templating engine: Something for you to abstract away your HTML from your code. Makes sense.

2. SQLAlchemy et al: An ORM. Fine.

3. A renderer. No idea.

4. JS libraries: JQuery et al: No idea what use these are except for adding pretty effects. How does this interact with the templating engine? How does this interact with the entire framework? Can I write code for jquery in Pyramid, or do I write JS separately, plug in my JS file into my template or...?

5. Form templating libraries (formish, formalchemy et al): How do these relate to the big picture? where do they plug in?

Any other important components that I'm missing?

So, could someone help me out and explain the stack?

Answer 1

1) A templating engine: Something for you to abstract away your HTML from your code. Makes sense.

There's several of these available. Mako tries to utilize many common Python idioms in the templates to avoid having to learn many new concepts. Jinja2 is similar to Django, but with more functionality. Genshi is if you like XML based templating.

As someone new to the whole thing, it's hard to say which is easiest to begin with unfortunately. Perhaps Jinja2.

2) SQLAlchemy et al. An ORM. Fine.

Yep.

3) A renderer. No idea.

A renderer is a Pyramid view configuration option, which tells Pyramid that if your view returns a dict, then it should be passed to the given 'renderer'. Renderers are setup to work with extension names, and Pyramid comes with several built-in: http://docs.pylonsproject.org/projects/pyramid/1.0/narr/renderers.html#built-in-renderers

In short, the renderer option merely looks at the name you pass it, and finds a template engine that matches the extension (.mak, .pt, 'json', 'string', .etc), and renders the dict results with it.

In many frameworks you don't designate a renderer as configuration, but instead have some code inside the view which looks something like this:

def somefunc(request):
    return render_to_response('/some/template.mak', {})

In Pyramid, you could do the same thing with:

@view_config(renderer='/some/template.mak')
def somefunc(request):
    return {}

There are several reasons the latter is a useful capability:

1. When it's entirely in configuration, you can override the renderer without having to change the view code logic.

2. You can add multiple configurations that change the renderer based on other conditions.

Consider this example which changes the renderer based on if the HTTP request is an XHR (AJAX request that wants a JSON formatted result, instead of a general HTTP request that wants HTML spit out by the template engine).

@view_config(renderer='json', xhr=True)
@view_config(renderer='/some/template.mak')
def somefunc(request):
    # lookup some_dict_data in a db, etc.
    return some_dict_data

4) JS libraries: JQuery et al. No idea what use these are except for adding pretty effects. How does this interact with the templating engine? How does this interact with the entire framework? Can I write code for jquery in pyramid, or do I write JS separately, plug in my JS file into my template or...?

JS libraries make it easier to write Javascript. They interact in the browser with the DOM, and have no interaction with Pyramid beyond sending HTTP requests to your web application that might want JSON formatted results.

To begin with, I'd suggest ignoring Javascript entirely until you're much more familiar with HTML, the DOM tree, and getting a site that works with just HTML, CSS, and the web-application.

5) Form templating libraries (formish, formalchemy et al) How do these relate to the big picture? where do they plug in?

I would highly suggest ignoring those entirely, and writing basic HTML form elements. You're new to the whole web stack, and there's really no need to jump straight to the most advanced aspects of web development without getting familiar with the basics first.

What you will need though, after writing basic forms, is you will want a form validation library that makes it easier to verify that the form which was submitted contains valid parameters. Back in the old days of PHP, people would write hundreds of lines of if/else statements that went through forms (some still do! ack!).

Nowadays we use form validation libraries which make it easy to declare what the valid parameters are for a form. I'd suggest FormEncode to begin with, as its fairly easy to use just for validation. For Pyramid, the easiest way to get going with FormEncode is probably pyramid_simpleform: http://packages.python.org/pyramid_simpleform/

For now, ignore the form rendering part and write the HTML form elements in the template yourself, and use pyramid_simpleform just for the easy FormEncode integration.

In short, start with just displaying HTML pages with links using view functions and templates (and use URL dispatch, its easier to grasp than traversal for beginners). Then add forms, their HTML and validation, then add CSS to start styling things.

Next you can start with some basic Javascript with jQuery to make things move around on the page, and work your way up to interacting with the webapp via AJAX to get more data. Just don't tackle too much at once, and it should be easier to see how they fit together.

Answer 2

3) A renderer. No idea.

Generally a renderer takes your data/model and converts it into something that the client wants. If the client is just a browser then the renderer will usually mash your data through a template to produce HTML. If the client is some JavaScript code or a non-browser application (a desktop application, another server that is consuming your data, ...) then the renderer would usually produce JSON (or possibly XML). You can think of this as a serialization or marshalling system.

4) JS libraries:

These are what you use to program the user interface. The user interface may just be some pretty effects slapped on top of HTML but it could be a lot more. Google Docs, for example, is JavaScript and a bit more than pretty effects; Cloud9 IDE would be another example full application built with JavaScript (thanks to Raynos for another example).

5) Form templating libraries

You can think of these as (more or less) macro systems for the template engine. If you have a data schema then you can use these things to generate template chunks and to automatically handle the server side processing of the corresponding return data.

Any other important components that I'm missing?

You can think of the modern web stack as a traditional client server system; this will probably anger some people but there's nothing radically new here except possibly the scale. The client is built with HTML and CSS for the layout and JavaScript (possibly with a toolkit) for the functionality and eye candy. The server is a web server of some sort. Communication between client and server is usually done in a combination of JSON and HTML over HTTP. You can think of web-1.0 (may deity forgive my marketing-talk terminology) as old school dumb terminals where web-2.0 is more like an X-terminal with some brains on the client.