Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

django-pipeline - Page load really slow

I am trying django-pipeline in order to minify static resources, use cache for them and make my templates simpler. My CSS and JS files are found and loaded by my browser but it takes about 10 seconds for my (very simple) home page to load.

enter image description here

I am using Python 2.7.6, Django 1.7.3 and django-pipeline 1.4.3. PyCharm runs the development server with a local virtualenv.

My settings.py contains the following:

DEBUG = True
TEMPLATE_DEBUG = DEBUG

INSTALLED_APPS = (
    'django_admin_bootstrapped', # custom admin
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # pip installed apps
    'pipeline',
    # project apps
    'myapp',
)

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'pipeline.middleware.MinifyHTMLMiddleware',
)

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'pipeline.finders.FileSystemFinder',
    'pipeline.finders.CachedFileFinder',
    'pipeline.finders.PipelineFinder',
)

STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'myapp/static'),
)

STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
PIPELINE_CSS_COMPRESSOR = 'pipeline.compressors.yuglify.YuglifyCompressor'
PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.yuglify.YuglifyCompressor'

PIPELINE_CSS = {
    'base': {
        'source_filenames': (
            'myapp/css/base.css',
            'myapp/bower_components/bootstrap/dist/css/bootstrap.css',
            'myapp/bower_components/Hover/css/hover.css',
            'myapp/bower_components/font-awesome/css/font-awesome.css',
        ),
        'output_filename': 'css/myapp.css',
    },
}

PIPELINE_JS = {
    'base': {
        'source_filenames': (
            'myapp/bower_components/jquery/dist/jquery.min.js',
            'myapp/bower_components/bootstrap/dist/js/bootstrap.min.js',
        ),
        'output_filename': 'js/myapp.js',
    },
}

My base HTML template contains the following:

{% load staticfiles %}
{% load pipeline %}

<!DOCTYPE html>
<html>
    <head>
        [...]
        {% block css %}
            {% stylesheet 'base' %}
        {% endblock css %}

        {% block javascript %}
            {% javascript 'base' %}
        {% endblock javascript %}

    </head>
    <body> [...] </body>
</html>

My home.html extends base.html but does not use the css nor javascript pipeline's template tags.

Just to make sure yuglify is available:

$ yuglify --version
0.1.4

What am I doing wrong here?

Note: browser does not find static assets (myapp.css and myapp.js) if PIPELINE_ENABLED = True.

like image 546
Q Caron Avatar asked Jan 30 '15 15:01

Q Caron


1 Answers

The problem is the template tag code is doing a bunch of stuff including running collectstatic for every request when debug is True which makes dev painfully slow. Even if debug is False the templatetag will still connect to and query S3 for several things. When the files are local this is not a (big) problem but when using S3 it is. The only solution I could come up with is writing my own simplified templatetag called pipelines.py.

Before you get into it there is two important things you need to know, first to get pipeline to work I have an empty shell S3PipelineStorage which combines pipeline and boto, you probably already have this if you have s3 + pipeline working but it is important:

from pipeline.storage import PipelineMixin
from storages.backends.s3boto import S3BotoStorage

class S3PipelineStorage(PipelineMixin, S3BotoStorage):
    pass

Then in settings:

STATICFILES_STORAGE = 'path.to.your.file.S3PipelineStorage'

Now, if you look down at the templatetag you will see I use staticfiles_storage.url similar to the original templatetag. This adds the s3 path to the relative path but if you don't add this setting you will query S3 every time to generate the URL. You can add the setting or just hard code your s3 path instead of staticfiles_storage.url however I suggest you add the setting because it will improve performance anywhere a URL is generated for an s3 resource.

AWS_S3_CUSTOM_DOMAIN = 'your_bucket-%s.s3.amazonaws.com' % ENVIRONMENT.lower()

Now you are ready for the templatetag. To use it simply {% load pipelines %} instead of {% load pipeline %}.

from django.contrib.staticfiles.storage import staticfiles_storage
from django import template
from django.template.loader import render_to_string
from django.utils.safestring import mark_safe
from pipeline.conf import settings

register = template.Library()

@register.simple_tag
def stylesheet(group):

    if group not in settings.PIPELINE_CSS:
        return ''

    if settings.DEBUG is False or settings.PIPELINE_ENABLED is True:
        context = {
            'type': 'text/css',
            'url': mark_safe(staticfiles_storage.url(settings.PIPELINE_CSS[group]['output_filename']))
        }
        html = render_to_string("pipeline/css.html", context)
    else:
        html = ''
        for path in settings.PIPELINE_CSS[group]['source_filenames']:
            context = {
                'type': 'text/css',
                'url': mark_safe(staticfiles_storage.url(path))
            }
            html = "%s\n        %s" % (html, render_to_string("pipeline/css.html", context))

    return html

@register.simple_tag
def javascript(group):

    if group not in settings.PIPELINE_JS:
        return ''

    if settings.DEBUG is False or settings.PIPELINE_ENABLED is True:
        context = {
            'type': 'text/javascript',
            'url': mark_safe(staticfiles_storage.url(settings.PIPELINE_JS[group]['output_filename']))
        }
        html = render_to_string("pipeline/js.html", context)
    else:
        html = ''
        for path in settings.PIPELINE_JS[group]['source_filenames']:
            context = {
                'type': 'text/javascript',
                'url': mark_safe(staticfiles_storage.url(path))
            }
            html = "%s\n        %s" % (html, render_to_string("pipeline/js.html", context))

    return html
like image 150
dotcomly Avatar answered Nov 08 '22 23:11

dotcomly