Pyramid: sessions and static assets

Question

Let me explain the problem:

I am serving my static assets via Pyramid:

config.add_static_view(name='static', path='/var/www/static')

And it works fine.

Now, I have a custom session factory that creates sessions in database. It checks if the browser presents a session cookie. If it does, it finds a session from the DB. If it does not, then a new session is created in DB and a cookie is returned to the browser.

So far so good.

Now, inside my home_view (that generates my home page), I do not access the request variable in any way:

@view_config(route_name='home', renderer="package:templates/home.mak")
def home_view(request):
    return {}

Because of this, what happens is when the user visits the home page, the session DOES NOT get created on the server. I think this is because Pyramid creates sessions lazily -- only when you access request.session. Hence, the response headers for the home page request DO NOT contain any Set-Cookie header for sessions.

Now inside my mako template for the home page, I am generating static URLs for JavaScript and CSS files...

<link rel="stylesheet" href="${request.static_url(...)}"

<script src="${request.static_url(...)}"></script>

Now, since I am serving the static assets from Pyramid, all the requests for these assets go through the entire Pyramid machinery.

So, what happens is when my browser sends requests to fetch static assets, Pyramid some how creates the session. That is, Pyramid is creating the session in the database and sending session cookie back when browser sends the requests for static assets. This is problem #1.

The browser sends all the requests for static assets in parallel. I am using the recent versions of Firefox and Chrome. Since the HTTP request for the actual HTML document did not return any Set-Cookie headers, the requests for static assets do NOT have any cookie headers. What this means is that Pyramid sees no session cookie for any of the requests, and it creates a new session in the database FOR EACH OF THE REQUESTS THAT IT GETS FOR THE STATIC ASSET.

If am fetching 7 static assets on my home page, and 7 session entries get created. This is because all these requests go in parallel to the server and none has session cookie, so Pyramid creates a session for each.

This problem does not arise if I deliberately access the session as part of the home page request. It creates a session in DB and sends a cookie to the browser which the browser then sends back for each static asset it requests from the server (in parallel).

@view_config(route_name='home', renderer="package:templates/home.mak")
def home_view(request):
    if request.session: pass
    return {}

How should I prevent the creation of sessions on static asset requests. Better yet, I would like Pyramid to not even touch the session factory when it receives a request for static asset -- is this possible?

Secondly, I don't understand WHY Pyramid is creating a new session on static requests?

UPDATE

Here is the session factory.

def DBSessionFactory(
        secret,
        cookie_name="sess",
        cookie_max_age=None,
        cookie_path='/',
        cookie_domain=None,
        cookie_secure=False,
        cookie_httponly=False,
        cookie_on_exception=True
    ):

    # this is the collable that will be called on every request
    # and will be passed the request
    def factory(request):
        cookieval = request.cookies.get(cookie_name)
        session_id = None
        session = None

        # try getting a possible session id from the cookie
        if cookieval is not None:
            try:
                session_id = signed_deserialize(cookieval, secret)
            except ValueError:
                pass

        # if we found a session id from  the cookie
        # we try loading the session
        if session_id is not None:
            # _load_session will return an object that implements
            # the partial dict interface (not complete, just the basics)
            session = _load_session(session_id)

        # if no session id from cookie or no session found
        # for the id in the database, create new
        if session_id is None or session is None:
            session = _create_session()

        def set_cookie(response):
            exc = getattr(request, 'exception', None)
            if exc is not None and cookie_on_exception == False:
                return
            cookieval = signed_serialize(session.session_id, secret)
            response.set_cookie(
                cookie_name,
                value=cookieval,
                max_age = cookie_max_age,
                path = cookie_path,
                domain = cookie_domain,
                secure = cookie_secure,
                httponly = cookie_httponly,
            )

        def delete_cookie(response):
            response.delete_cookie(
                cookie_name,
                path = cookie_path,
                domain = cookie_domain,
            )

        def callback(request, response):
            if session.destroyed:
                _purge_session(session)
                delete_cookie(response)
                return

            if session.new:
                set_cookie(response)

            # this updates any changes to the session
            _update_session(session)


        # at the end of request
        request.add_response_callback(callback)

        # return the session from a call to the factory
        return session

    # return from session factory
    return factory

And then,

factory = DBSessionFactory('secret')
config.set_session_factory(factory)

UPDATE

My custom authentication:

class RootFactory:
    __acl__ = [
        (Allow, Authenticated, 'edit'),

        # only allow non authenticated users to login
        (Deny, Authenticated, 'login'),
        (Allow, Everyone, 'login'),
    ]

    def __init__(self, request):
        self.request = request



class SessionAuthenticationPolicy(CallbackAuthenticationPolicy):
    def __init__(self, callback=None, debug=False):
        self.callback = callback
        self.debug = debug

    def remember(self, request, principal, **kw):
        return []

    def forget(self, request):
        return []

    def unauthenticated_userid(self, request):
        if request.session.loggedin:
            return request.session.userid
        else:
            return None

And then,

config.set_root_factory(RootFactory)

config.set_authentication_policy(SessionAuthenticationPolicy())
config.set_authorization_policy(ACLAuthorizationPolicy())

Answer 1

I'm unable to reproduce this behavior in a dummy project which leads me to believe that you have some configuration affecting things that isn't shown here. Clearly if any authentication is invoked a session will be created, as per your authentication policy. Static assets are (by default) registered with NO_PERMISSION_REQUIRED which means that they will not invoke any of the authentication APIs within Pyramid (and I've verified that this is the case).

Requests for static assets do invoke the entire request pipeline, meaning that if you have any code in any subscribers, or your root factory that invoke has_permission or other security APIs, or touch the session directly themselves, then this would explain the behavior you're seeing since your sessions are coupled to your authentication.

Answer 2

Here is a dummy project to reproduce the problem:

1. setup a virtualenv environment and install Pyramid in it.

2. Install a starter project: pcreate -s starter IssueApp

3. Delete all the unnecessary files so that you have this simple tree:

Tree

.
├── CHANGES.txt
├── development.ini
├── issueapp
│   ├── __init__.py
│   └── static
│       └── pyramid.png
├── README.txt
└── setup.py

Note that we wil write the entire app in the __init__.py file -- so everything else is removed.

Now install the project: (env) $ python setup.py develop This will install your project into virtual environment.

The development.ini file:

[app:main]
use = egg:IssueApp#main

pyramid.reload_all = true
pyramid.reload_templates = true
pyramid.debug_all = true
pyramid.debug_notfound = true
pyramid.debug_routematch = true
pyramid.prevent_http_cache = true
pyramid.default_locale_name = en

[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 7777

[loggers]
keys = root, issueapp

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = INFO
handlers = console

[logger_issueapp]
level = INFO
handlers =
qualname = issueapp

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s

The __init__.py file:

from pyramid.config import Configurator

from pyramid.view import view_config
from pyramid.response import Response

from pyramid.authentication import CallbackAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy

from pyramid.security import (
    Allow, Deny,
    Everyone, Authenticated,
)


def main(global_config, **settings):
    """ This function returns a Pyramid WSGI application.
    """
    config = Configurator(settings=settings)

    #config.add_static_view('static', 'static', cache_max_age=3600)
    config.add_static_view(name='static', path='issueapp:static')
    config.add_route('home', '/')

    config.set_root_factory(RootFactory)
    config.set_authentication_policy(DummyAuthPolicy())
    config.set_authorization_policy(ACLAuthorizationPolicy())

    config.scan()
    return config.make_wsgi_app()


@view_config(route_name='home')
def home_view(request):
    src = request.static_url('issueapp:static/pyramid.png')
    return Response('<img src='+ src + '>')


class RootFactory:
    __acl__ = [
        (Allow, Authenticated, 'edit'),
        (Deny, Authenticated, 'login'),
        (Allow, Everyone, 'login'),
    ]

    def __init__(self, request):
        self.request = request


class DummyAuthPolicy(CallbackAuthenticationPolicy):
    def __init__(self, callback=None, debug=False):
        self.callback = callback
        self.debug = debug

    def remember(self, request, principal, **kw):
        return []

    def forget(self, request):
        return []

    def unauthenticated_userid(self, request):
        # this will print the request url
        # so we can know which request is causing auth code to be called            
        print('[auth]: ' + request.url)

        # this means the user is authenticated
        return "user"

Now run the app

pserve  development.ini  --reload
Starting subprocess with file monitor
Starting server in PID 2303.
serving on http://0.0.0.0:7777

Finally, clear all history from your browser (this is important or the issue might not reveal itself) and access the page. this gets printed on the console:

[auth]: http://192.168.56.102:7777/static/pyramid.png   

Which shows that auth code is getting called for static requests.

Now, when I set the log level to DEBUG, this is the output of console on accessing the page:

pserve  development.ini  --reload
Starting subprocess with file monitor
Starting server in PID 2339.
serving on http://0.0.0.0:7777
2013-03-27 03:40:55,539 DEBUG [issueapp][Dummy-2] route matched for url http://192.168.56.102:7777/; route_name: 'home', path_info: '/', pattern: '/', matchdict: {}, predicates: ''
2013-03-27 03:40:55,540 DEBUG [issueapp][Dummy-2] debug_authorization of url http://192.168.56.102:7777/ (view name '' against context ): Allowed (no permission registered)
2013-03-27 03:40:55,685 DEBUG [issueapp][Dummy-3] route matched for url http://192.168.56.102:7777/static/pyramid.png; route_name: '__static/', path_info: '/static/pyramid.png', pattern: 'static/*subpath', matchdict: {'subpath': ('pyramid.png',)}, predicates: ''
[auth]: http://192.168.56.102:7777/static/pyramid.png
2013-03-27 03:40:55,687 DEBUG [issueapp][Dummy-3] debug_authorization of url http://192.168.56.102:7777/static/pyramid.png (view name '' against context ): ACLDenied permission '__no_permission_required__' via ACE '' in ACL [('Allow', 'system.Authenticated', 'edit'), ('Deny', 'system.Authenticated', 'login'), ('Allow', 'system.Everyone', 'login')] on context  for principals ['system.Everyone', 'system.Authenticated', 'user']

---

Note that the [auth]: ... message is getting printed ONLY ONCE -- for the static asset request, and NOT for the home page request. This is strange because it means that the auth policy is consulted for static assets but not for normal requests. (Unless of course there is a permission involved, which in my view isn't).