GithubHelp home page GithubHelp logo

jazzband / django-pipeline Goto Github PK

View Code? Open in Web Editor NEW
1.5K 27.0 373.0 1.29 MB

Pipeline is an asset packaging library for Django.

Home Page: https://django-pipeline.readthedocs.io/

License: MIT License

Python 95.52% HTML 1.05% CSS 1.13% JavaScript 1.26% CoffeeScript 0.06% LiveScript 0.02% Less 0.02% SCSS 0.06% Stylus 0.01% EJS 0.20% Jinja 0.51% TypeScript 0.16%

django-pipeline's Introduction

Pipeline

Jazzband GitHub Actions Coverage Documentation Status

Pipeline is an asset packaging library for Django, providing both CSS and JavaScript concatenation and compression, built-in JavaScript template support, and optional data-URI image and font embedding.

Django Pipeline Overview

Installation

To install it, simply:

pip install django-pipeline

Quickstart

Pipeline compiles and compress your assets files from STATICFILES_DIRS to your STATIC_ROOT when you run Django's collectstatic command.

These simple steps add Pipeline to your project to compile multiple .js and .css file into one and compress them.

Add Pipeline to your installed apps:

# settings.py
INSTALLED_APPS = [
    ...
    'pipeline',
]

Use Pipeline specified classes for STATICFILES_FINDERS and STATICFILES_STORAGE:

STATICFILES_STORAGE = 'pipeline.storage.PipelineManifestStorage'

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'pipeline.finders.PipelineFinder',
)

Configure Pipeline:

# The folowing config merges CSS files(main.css, normalize.css)
# and JavaScript files(app.js, script.js) and compress them using
# `yuglify` into `css/styles.css` and `js/main.js`
# NOTE: Pipeline only works when DEBUG is False
PIPELINE = {
    'STYLESHEETS': {
        'css_files': {
            'source_filenames': (
                'css/main.css',
                'css/normalize.css',
            ),
            'output_filename': 'css/styles.css',
            'extra_context': {
                'media': 'screen,projection',
            },
        },
    },
    'JAVASCRIPT': {
        'js_files': {
            'source_filenames': (
                'js/app.js',
                'js/script.js',
            ),
            'output_filename': 'js/main.js',
        }
    }
}

Then, you have to install compilers and compressors binary manually.

For example, you can install them using NPM and address them from node_modules directory in your project path:

PIPELINE.update({
    'YUGLIFY_BINARY': path.join(BASE_DIR, 'node_modules/.bin/yuglify'),
})
# For a list of all supported compilers and compressors see documentation

Load static files in your template:

{% load pipeline %}
{% stylesheet 'css_files' %}
{% javascript 'js_files' %}

Documentation

For documentation, usage, and examples, see: https://django-pipeline.readthedocs.io

Issues

You can report bugs and discuss features on the issues page.

Changelog

See HISTORY.rst.

django-pipeline's People

Contributors

ara818 avatar asfaltboy avatar blankser avatar bryanchow avatar camilonova avatar chipx86 avatar cyberdelia avatar davidt avatar dcharbonnier avatar dnerdy avatar estebistec avatar fdintino avatar hsiaoyi0504 avatar jezdez avatar jwhitlock avatar kevinmarsh avatar kylemacfarlane avatar leonardoo avatar miki725 avatar nasirhjafri avatar pdr avatar pelme avatar peymanslh avatar pre-commit-ci[bot] avatar rdil avatar sassanh avatar slafs avatar streeter avatar svetlyak40wt avatar vbabiy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

django-pipeline's Issues

Uncaught ReferenceError: xxx is not defined

We have following js structure - all js are loaded to user at once, except page-specific initializers ( e.g. init.main.js , init.subscription.js, init.product_list.js ). So usual structure looks like :

{% compressed_js 'all' %}
<script src='init.product_list.js'></script> 

init.product_list.js tries to call functions from 'all', but cannot do it, as compressed_js encapsulates everything inside

 (function($){...}).call(this)

Demonstration -http://jsfiddle.net/TwSqe/11/

Is it possible to solve this issue without modifying code ?

PIPELINE_VERSION = False doesn't work

I prefer to leave filename-based versioning off, because we need to have our output files in version control and removing old compressed files is tedious.

When PIPELINE_VERSION is set False, the compressed files are not overwritten, but new files are added named with _1, _2, _3 extensions. The templatetags do not read these files, but read the original compressed file, so changes are not displayed.

I'm using YUICompressor for compilation of both css and js.

Cleanup doesn't work with S3

Line 53 in pipeline.versions.Versioning.cleanup calls storage.exists on a directory.

S3 however has no concept of directories, only files. So exists('css') will return False while exists('css/common.r123.css') could return True.

I think the better solution would be to just go straight to listdir and catch an exception if it doesn't work.

Windows support

It would be nice to support Windows as there are those of us who do develop on it. It would have also been nice to have seen that the app does not support it so I didn't have to start digging in the code.

Use staticfiles to find files

When using contrib.staticfiles with runserver you don't need to run the collectstatic command to actually serve the files. This is useful when developing as it can serve the files straight out of your app directory.

Pipeline on the other hand only looks in PIPELINE_ROOT and either throws an exception if the files don't exist or compresses outdated versions of the file. The only real solution is to run collectstatic every time you alter your files, which of course is a huge hassle.

For the compressors you can get round this problem by setting PIPELINE = False when developing but for the compilers this isn't any good, since your code obviously needs to be compiled.

If available, I think that Pipeline should use the staticfiles finders to locate files.

Caught CompilerError while rendering

I'm getting a really weird error when trying to render my pages:

Environment:


Request Method: GET
Request URL: http://localhost:8000/

Django Version: 1.3
Python Version: 2.7.1
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.staticfiles',
 'pipeline',
 'reversion',
 'south',
 'myproject.web']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Template error:
In template /home/rfkrocktk/Desktop/myproject/src/myproject/web/templates/desktop/root.html, error at line 6
   Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:20:in `read': No such file or directory - -x (Errno::ENOENT)
    from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:20
    from /usr/local/bin/lessc:19:in `load'
    from /usr/local/bin/lessc:19

   1 : {% load static %}


   2 : {% load compressed %}


   3 : <!DOCTYPE html>


   4 : <html>


   5 :  <head>


   6 :      {% block stylesheets %} {% compressed_css 'all' %} {% endblock %}


   7 :      <title>{% block title %}{% endblock %}&nbsp;&raquo;&nbsp;myproject</title>


   8 :  </head>


   9 :  <body>


   10 :         {% block body %}{% endblock %}


   11 :         {% block scripts %}{% compressed_js 'all' %}{% endblock %}


   12 :     </body>


   13 : </html>


   14 : 

Traceback:
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/views/generic/simple.py" in direct_to_template
  28.     return HttpResponse(t.render(c), mimetype=mimetype)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  123.             return self._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  127.         return compiled_parent._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  127.         return compiled_parent._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  64.             result = block.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/templatetags/compressed.py" in render
  30.             package['paths'] = self.packager.compile(package['paths'])
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/packager.py" in compile
  50.         return self.compiler.compile(paths)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/compilers/__init__.py" in compile
  25.                         compiled_content = compiler.compile_file(content)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/compilers/less/__init__.py" in compile_file
  25.         content = self.execute_command(command, content)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/compilers/__init__.py" in execute_command
  86.             raise CompilerError(error)

Exception Type: TemplateSyntaxError at /
Exception Value: Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:20:in `read': No such file or directory - -x (Errno::ENOENT)
    from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:20
    from /usr/local/bin/lessc:19:in `load'
    from /usr/local/bin/lessc:19

I'm on Ubuntu 11.04. I installed less via: gem install less then symlinked lessc it to /usr/local/bin/lessc so it could find it. It seems like it's calling the compiler in a weird way. Is this a known bug or am I doing something wrong?

Less compiler not respecting relative paths with @import

I'm running django-pipeline version 1.1.22 (latest), and the following isn't working yet in .less files:

 @import "bootstrap-1.4/bootstrap.less";

The reason one would do this would be to include the Less library and be able to predefine variables and be able to use the mixins exposed by the library in one's own Less code. For instance:

@basefont: 14px; /* in bootstrap, it's 13 by default, so we override it before it's imported */

@import "bootstrap-1.4/bootstrap.less";

.ghostly { /* we use bootstrap's defined .opacity mixin to create this effect. */
    .opacity(90);
}

Currently, this doesn't work and yields:

Environment:


Request Method: GET
Request URL: http://localhost:8000/

Django Version: 1.3
Python Version: 2.7.1
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.staticfiles',
 'django_bcrypt',
 'pipeline',
 'reversion',
 'south',
 'myproject.web']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Template error:
In template /home/rfkrocktk/Desktop/myproject/src/myproject/web/templates/desktop/root.html, error at line 6
   Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
    from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
    from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
    from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
    from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
    from /usr/local/bin/lessc:19:in `load'
    from /usr/local/bin/lessc:19

   1 : {% load static %}


   2 : {% load compressed %}


   3 : <!DOCTYPE html>


   4 : <html>


   5 :  <head>


   6 :      {% block stylesheets %} {% compressed_css 'all' %} {% endblock %}


   7 :      <title>{% block title %}{% endblock %}&nbsp;&raquo;&nbsp;myproject</title>


   8 :  </head>


   9 :  <body>


   10 :         {% block body %}{% endblock %}


   11 :         {% block scripts %}{% compressed_js 'all' %}{% endblock %}


   12 :     </body>


   13 : </html>


   14 : 

Traceback:
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/views/generic/simple.py" in direct_to_template
  28.     return HttpResponse(t.render(c), mimetype=mimetype)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  123.             return self._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  127.         return compiled_parent._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  127.         return compiled_parent._render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in _render
  117.         return self.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/loader_tags.py" in render
  64.             result = block.nodelist.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/base.py" in render
  744.                 bits.append(self.render_node(node, context))
File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/template/debug.py" in render_node
  73.             result = node.render(context)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/templatetags/compressed.py" in render
  30.             package['paths'] = self.packager.compile(package['paths'])
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/packager.py" in compile
  50.         return self.compiler.compile(paths)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in compile
  25.                         compiled_content = compiler.compile_file(content, storage.path(path))
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/less/__init__.py" in compile_file
  17.         content = self.execute_command(command)
File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.22-py2.7.egg/pipeline/compilers/__init__.py" in execute_command
  88.             raise CompilerError(error)

Exception Type: TemplateSyntaxError at /
Exception Value: Caught CompilerError while rendering: /var/lib/gems/1.8/gems/less-2.0.7/lib/less/loader.rb:46:in `error': file 'bootstrap-1.4/bootstrap.less' wasn't found. (Less::ParseError)
    from at Function.importer (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/index.js:114:13)
    from at Object.push (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:73:25)
    from at Object.<anonymous> (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/tree/import.js:30:17)
    from at Object.import (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:953:28)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.directive (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:967:29)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.primary (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:428:80)
    from at $ (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:103:24)
    from at Object.parse (/var/lib/gems/1.8/gems/less-2.0.7/lib/less/js/lib/less/parser.js:264:42)
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:65:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:13:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:27:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:25:in `lock'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:12:in `calljs'
    from /var/lib/gems/1.8/gems/less-2.0.7/lib/less/parser.rb:63:in `parse'
    from /var/lib/gems/1.8/gems/less-2.0.7/bin/lessc:22
    from /usr/local/bin/lessc:19:in `load'
    from /usr/local/bin/lessc:19

Less compiler don't support @import statements

A pretty simple test for django-pipeline is having it compile and minify the bootstrap library.

Unfortunately, it seems that since the Less compiler moves files to /tmp first and then tries to compile them, this breaks things, as if something is in a directory and it refers to a local file.

For example, given bootstrap.less:

@import "reset.less";
@import "variables.less";
@import "mixins.less";
@import "scaffolding.less";
@import "type.less";
@import "forms.less";
@import "tables.less";
@import "patterns.less";

compilation will fail, as it's trying to compile things from the copied file in the /tmp directory, rather than from where it lives on the filesystem.

Support TEMPLATE_LOADERS for loading JavaScript Templates

Feature request - from an organizational standpoint, JavaScript templates should at least have the option to be stored in the same directories are regular templates. I wanted to open this ticket for discussion/concerns before I attempt a patch.

Problem with folder wildcards when files in parent folder

Say you have your pipeline set up to look for sources in any of the subfolders of the given path, e.g. "js/ * / *.js".
This works well when there are no files in the "js" folder, but it breaks when the folder also contains files.
The cause is in pipeline.glob.
glob1 returns a combined list of all the files and directories. iglob uses this list to scan folders recursively when it needs to. The problem is that it executes glob1 on files as well as folders, and glob1 causes an exception when attempted to run on non-folders.

There should be separate roots for source and output files.

According to http://readthedocs.org/docs/django-pipeline/en/1.1.23/configuration.html#specifying-files :

Note that all filenames are specified relative to PIPELINE_ROOT, and thus the source files needs to be in your PIPELINE_ROOT.

I believe that having to keep source and output files in the same location (PIPELINE_ROOT) is a very bad restriction. PIPELINE_ROOT is a directory that gets published. If source files need to be stored in the same location, that means that all source files get published as well, which is not what we normally want.

This restriction is especially bad if we like to keep our source files within our app folders, like this:

/myproject/myapp/css/...
/myproject/myapp/js/...
/myproject/myapp/templates/...
/myproject/myapp/models.py
/myproject/myapp/views.py

In order to do this with current version of Pipeline, you'd have to publish the entire myapp folder, together with all .py files... So Pipeline forces you to move the CSS and JS files somewhere else.

And even then the generated output files get stored in the same location as the source files. I do not like to pollute my source folders with generated files, and again it's not possible to keep them separate.

(By the way, even if you try to trick Pipeline by using absolute paths, an exception is thrown unless all files are contained within PIPELINE_ROOT.)

For all of the above reasons, source and output roots really should be kept separate.

Get rid of PIPELINE_AUTO

Since compiling and compressing assets in the request/response loop is slow and dangerous, even with cache.
And PIPELINE_AUTO is also a confusing settings, pipeline should only work with two mode :

  • DEBUG mode : which compile and link each single asset, for easy development.
  • PRODUCTION mode : which only look for latest compiled, compressed assets file.

KeyError: 'templates' if I use synccompress with external_urls

Here's the configuration I use:

DEBUG = False

PIPELINE_VERSION = True

PIPELINE_JS = {
   'live': {
    'external_urls': ( 'http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js', ),
  },
}

PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.yui.YUICompressor'
PIPELINE_YUI_BINARY = 'java -jar ' + os.path.join(PROJECT_PATH, 'external/yuicompressor-2.4.7.jar')

And here's the traceback I receive when I run 'python manage.py synccompress live':

  File "/Users/macuser/projects/envs/balaur/lib/python2.6/site-packages/pipeline/management/commands/synccompress.py", line 50, in handle
    packager.pack_javascripts(package, sync=sync, force=force)
  File "/Users/macuser/projects/envs/balaur/lib/python2.6/site-packages/pipeline/packager.py", line 76, in pack_javascripts
    return self.pack(package, self.compressor.compress_js, js_compressed, templates=package['templates'], **kwargs)
KeyError: 'templates'

Infinite loop when used in conjunction with collectstatic --link

Using django-pipeline 1.1.25 to generate compressed css and js to STATIC_ROOT along with python manage. py collectstatic --link (django 1.3.1) will cause an infinite loop with no errors when rendering a template with {% compressed_js %} or {% compressed_css %} if new css or js needs to be generated. This issue will be fixed with the addition of the --clear option to collectstatic in django 1.4 ( https://code.djangoproject.com/changeset/16509)

In the meantime the solution is to use PIPELINE_AUTO = False, delete your generated css and js files before running
python manage.py collectstatic --link and then run python manage.py synccompress to regenerate your compressed css and js.

I'm just reporting this form information purposes really, because it took me a while to debug so could possibly help others who are scratching their heads in a similar situation.

IOError from execute_command in compressors/init.py

Hi guys, I have had a look at this but I still dont exactly know what the problem I am experiencing is coming from.

I am configured using yui but Django throws a TemplateError with a IOError: (22, 'Invalid argument') at compressors/init.py line 235 (pipe.stdin.write(content)). The Django template error :

TemplateSyntaxError at /
Caught IOError while rendering: [Errno 22] Invalid argument.

Exception Type: TemplateSyntaxError
Exception Value:Caught IOError while rendering: [Errno 22] Invalid argument"

Here is the template code :

{% load compressed %}

{% compressed_css 'template' %}

Here is the settings definition which uses django static files.

PIPELINE_CSS = {
    'template': {
        'source_filenames': (
            'css/template/reset.css',
            'css/template/default.css',
            'css/project/account_pages.css',
            'css/project/calculator.css',
            'css/project/contact.css',
            'css/project/results.css',
            'css/jquery-ui/Aristo/aristo.css'
        ),
        'output_filename': 'CACHE/css/template.css',
    }
}

A breakpoint before this line yields the following in pdb.

(Pdb) command
'C:/Documents and Settings/mark/Documents/project/bin/yuicompressor.jar --type=css '

(Pdb) content
'/* STYLES */\n#navigation {\n    height: 32px; \n}\n.nav{\n   ...'

I get the same error when I push the changes to my Linux VPS. I am thinking YUI isnt reading stdin properly but you guys are obviously using this with no issues. I have tried YUI 2.4.2 and all the way up to 2.4.7. I have tried removing much of the content being compressed but still receive the error. If I switch to CSSTidy, CSS compression works but fails with the same error using YUI for Javascript.

Any help or suggestions would be appreciated.

Improve current version calculation

Improve current version calculation in version_from_file to avoid scanning directory :

def version_from_file(self, path, filename):
    filename = settings.PIPELINE_VERSION_PLACEHOLDER.join([re.escape(part) for part in filename.split(settings.PIPELINE_VERSION_PLACEHOLDER)])
    regex = re.compile(r'^%s$' % self.output_filename(filename, r'([A-Za-z0-9]+)'))
    versions = []
    for f in sorted(storage.listdir(path)[1], reverse=True):
        version = regex.match(f)
        if version and version.groups():
            versions.append(version.group(1))
    versions.sort()
    return versions[-1]

Document that javascript is wrapped in a closure

When a compressor has squashed everything it is not obvious nor documented that you have also wrapped them in a closure. previously working functions are not accessible from global scope.

I'm sure its a very good practice to write functions with a specific scope

But its also a very good practice to document that your compiler is doing this to us ;)

thanks ! great project anyway !

Asset path normalization should be opt-out

Asset paths in stylesheets are rewritten to be absolute, but this isn't always beneficial. Distributable webapps with processed stylesheets that can be installed either at the root of a domain or within a subdirectory will break if an absolute path is used. Some behavior for opting out of this feature is critical for such cases.

I'll be happy to provide a patch to address this.

external_urls should not be alone in groups

It would be useful to have 'source_filenames' and 'external_urls' options in one Javascript group. It would be more pragmatic and allow to easier organizing code in templates.

Add support for pyScss

Add a compiler with support for pyScss: https://github.com/Kronuz/pyScss

I don't want to install ruby, so I wrote my own compiler that uses pyScss to compile my .scss files. It works nicely, so it would be great if it were added as a default compiler.

import os, fnmatch
import scss

from django.conf import settings
from django.contrib.staticfiles import finders

from pipeline.compilers import SubProcessCompiler

def finder(glob):
    for finder in finders.get_finders():
        for path, storage in finder.list([]):
            if fnmatch.fnmatchcase(path, glob):
                yield path, storage

# this is where pyScss looks for images and static data
scss.STATIC_ROOT = finder
scss.STATIC_URL = settings.STATIC_URL

# this is where pyScss outputs the generated/compiled files
scss.ASSETS_ROOT = os.path.join(settings.MEDIA_ROOT, 'assets/')
scss.ASSETS_URL = settings.MEDIA_URL + 'assets/'

class PyScssCompiler(SubProcessCompiler):
    output_extension = 'css'

    def match_file(self, filename):
        return filename.endswith('.scss')

    def compile_file(self, content, path):
        compiler = scss.Scss(scss_opts={
            'debug_info': settings.DEBUG,
        })
        return compiler.compile(open(path).read())

Regression: Paths to STATIC_ROOT must now end in os.sep

I've recently updated to the current stable version of django-pipeline and this has broken my Django app. I've traced down the problem to changes to how URLs are generated, I believe in this version: 074c7b4

My old Django settings.py read as follows:

STATIC_ROOT = os.path.join(PROJECT_ROOT, 'compiled_media')

however due to the relpath call, the actually directory name gets lost (because it doesn't end in a /), and I now need to use:

STATIC_ROOT = os.path.join(PROJECT_ROOT, 'compiled_media') + os.sep

This is a regression from previous versions and should ideally be fixed (this is likely to impact a number of users of Molly which has this project as a dependency)

Javascript template names get an _ prefix on Windows

On Windows, if you have a template file called mytemp.jst, the resulting template will get named "_mytemp", i.e. the javascript template variable will get stored into window.JST['_mytemp'].

This happens because the pipeline.glob.iglob functions uses os.path.join, which does this:
"js/templates" + "mytemp.jst" = "js/templates\mytemp.jst" (notice the backslash).
When this path reaches pipeline.compressors.template_name, it strips the base path and any forward slashes, but not the backslashes:

name = re.sub(r"^%s\/?(.*)%s$" % ( ...

Subsequently, it replaces all slashes and backslashes with underscores, so "\mytemp.jst" results into "_mytemp".

This is especially troublesome because it generates different names on Windows than on a Unix.

Absolute path for asset

It turns absolute paths into relative paths, i.e. the exact opposite of what it's meant to do.

Imagine the following CSS in http://example.com/css/common.css

body {
    background-image: url(/img/bg.jpg);
}

This should load the image http://example.com/img/bg.jpg.

However, since the original code here only matches double slashes it doesn't realise that this an absolute path and turns it into the following:

body{
    background-image:url(img/bg.jpg);
}

Which of course tries to load the file from http://example.com/css/img/bg.jpg.

manage.py synccompress - No such file or directory

I've got a pretty simple application configuration going here:

STATIC_URL = '/static/'
STATIC_ROOT = '/tmp/myproject/static/'
STATICFILES_FINDERS = (
    'pipeline.finders.PipelineFinder',
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)

# ...
PIPELINE = not DEBUG
PIPELINE_AUTO = True
PIPELINE_VERSION = True

PIPELINE_COMPILERS = (
    'pipeline.compilers.less.LessCompiler',
)

PIPELINE_CSS = {
    'all': {
        'source_filenames': (
            'styles/*.less',
        ),
        'output_filename': 'styles/application.?.css',
    }
}

PIPELINE_JS = {
    'all': {
        'source_filenames': (
            'scripts/jquery-1.6.2.min.js',
            'scripts/bootstrap-*.js',
            'scripts/jquery.*.js',
            'scripts/application.js',
        ),
        'output_filename': 'scripts/application.?.js',
    }
}

# ...

INSTALLED_APPS = (
    # ...
    'pipeline',
    'myproject.web',
)

When I go to run the synccompress task, I get the following:

Traceback (most recent call last):
  File "bin/django", line 22, in <module>
    djangorecipe.manage.main('myproject.settings')
  File "/home/rfkrocktk/Desktop/myproject/eggs/djangorecipe-0.99-py2.7.egg/djangorecipe/manage.py", line 16, in main
    management.execute_manager(mod)
  File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/rfkrocktk/Desktop/myproject/eggs/Django-1.3-py2.7.egg/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/management/commands/synccompress.py", line 39, in handle
    packager.pack_stylesheets(package, sync=sync, force=force)
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/packager.py", line 47, in pack_stylesheets
    variant=variant, **kwargs)
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/packager.py", line 55, in pack
    package['output'], package['paths'])
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/versioning/__init__.py", line 45, in need_update
    version = self.version(paths)
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/versioning/__init__.py", line 20, in version
    return getattr(self.versioner, 'version')(paths)
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/versioning/mtime/__init__.py", line 11, in version
    [int(time.mktime(storage.modified_time(path).timetuple())) for path in paths]
  File "/home/rfkrocktk/Desktop/myproject/eggs/django_pipeline-1.1.18.1-py2.7.egg/pipeline/storage.py", line 29, in modified_time
    return datetime.fromtimestamp(os.path.getmtime(self.path(name)))
  File "/usr/lib/python2.7/genericpath.py", line 54, in getmtime
    return os.stat(filename).st_mtime
OSError: [Errno 2] No such file or directory: '/tmp/myproject/static/s'

What am I doing wrong here?

Add conditional compilation for Less while DEBUG = True

Since Less gives the option of compilation at runtime via a JavaScript library, it'd be nice if django-pipeline could include an option for using runtime compilation while DEBUG = True.

Currently, you can do this with the following template code, but it'd be nicer if this could be integrated with django-pipeline:

{% if debug %}
    <link href="{% get_static_prefix %}styles/application.less" rel="stylesheet" type="text/css" />
    <script src="{% get_static_prefix %}scripts/less.js" type="text/javascript"></script>
{% else %}
    {% compressed_css 'all' %}
{% endif %}

Autocompilation of LESS to CSS without collectstatic/synccompress

It'd be awesome if whenever *.less files were changed while DEBUG = True, django-pipeline could recompile them to create a more seamless experience in development.

Currently, if I add a line to a .less file, I have to then run manage.py collectstatic to move the files to the right folder then manage.py synccompress to compile the less to CSS.

PIPELINE_ROOT should not default to STATIC_ROOT

Hello, I found this rather confusing :
PIPELINE_JS expect files relative to PIPELINE_ROOT = STATIC_ROOT by default.
But STATIC_ROOT is where static files are collected by manage collectstatic.
One should never put one's source files there !

Add integration with CSSJanus (automatic LTR / RTL swap)

Some great Python tools like CSSJanus helps developers to swap their stylesheets from LTR to RTL (or the other way). Making it usable with django-pipeline today can really be "hackish".

The idea would be to generate each time two versions of each compressed CSS file : one LTR, and one RTL (with CSSJanus). When using the compressed_js tag, the file path would be translated to the correct version of the file, based on the context.

Problem with glob syntax and coffeescript

When using the glob syntax to add coffeescript files, and PIPELINE = False, the resulting .js files are generated and added to the template via {% compressed_js 'application' %}, but in subsequents calls(hit refresh) it would not add the js references generated from the coffeescript globs.

Current settings:

PIPELINE = False

STATIC_ROOT = os.path.join(PROJECT_ROOT, 'collected/')

STATICFILES_DIRS = (
       os.path.join(PROJECT_ROOT, 'static/'),
)

STATICFILES_FINDERS = (
   'pipeline.finders.PipelineFinder',
   'django.contrib.staticfiles.finders.FileSystemFinder',
   'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)

PIPELINE_STORAGE = 'pipeline.storage.PipelineFinderStorage'

PIPELINE_JS = {
    'application': {
        'source_filenames': (
          'js/app.coffee',
          'js/app/models/*.coffee',
          'js/app/collections/*.coffee',
          'js/app/templates/**/*.jst',
          'js/setup.coffee',
        ),
        'output_filename': 'js/webapp.js',
    }
}

In this case, it would return app.js, the templates and setup.js, but not anything that matches js/app/models/*.coffee or js/app/collections/*.coffee.

It seems to be a problem with PipelineFinderStorage.

Image version

Would love to see image versioning a la django-mediagenerator. Any plans to do something like this?

lessc on Windows

I'd like to know how I can make Django-pipeline work with LessCSS on Windows.

lessc is installed:

C:\>lessc
    dotless Compiler 1.1.0.7
    Compiles .less files to css files.

Django Pipeline Settings:

COMPRESS_PRECOMPILERS = (
    ('text/less', 'lessc {infile} {outfile}'), )

Error:

Caught FilterError while rendering: 'lessc' is not recognized as an internal or external command,
operable program or batch file.

JST Concatenation Strategy

Currently, the pipeline documentation points out that given a template like js/templates/photo/detail.jst, it would be exposed as JST.photo_detail(). But as you can see, that's not the way it is working right now. The output is the JST object as a hashtable, where the key is the path of the file, using _ instead of / as path separators.

I think the current(but undocumented) way to do it is better, but it should use / instead: JST['js/templates/photo/detail.jst']. Or better yet, do like jammit does:

When including templates from different directories, the common prefix is ignored, and the rest of the path becomes the name of the template:

javascripts:
  workspace:
    - app/views/accounts/badge.jst
    - app/views/common/dialog.jst
    - app/views/common/menu.jst

Then, from your JavaScript:

  JST['accounts/badge']
  JST['common/dialog']
  JST['common/menu']

Do you agree with this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.