Integrate with Sprockets about less-rails

[metaskills]

@import should have nothing to do with sprockets or tilt. Only places where sprockets gets involved is in the directive processors at the top of each file. After that, each template is rendered from right to left with each extension. In the case with less, @imports are supported to other files ending with .less extension.

If this is not the case can you show me in the code where with some examples of how this works in something like Sass/Compass?

[stouset]

If this is not the case can you show me in the code where with some examples of how this works in something like Sass/Compass?

Be glad to once I get to my laptop.

The biggest issue is that Sprockets uses its internal representation of the import hierarchy to determine staleness. This means that updated Less partials won't take effect until you make a change to the topmost stylesheet.

I suspect that this will also prevent Sprockets from updating the asset tag hash unless the topmost stylesheet has been changed as well.

[stouset]

From the Compass issue on Rails 3.1 support:

What we'll do is make Sass @import directives find things via sprockets. Sass users shouldn't have to use hacky comments to accomplish this behavior. Sass has already implemented this feature, we just need to glue it together.

[stouset]

Another issue it that Sprockets will serve you partials individually in development for ease of debugging. When @import is processed independently in Less, Sprockets can't do this. So, without integration with Sprockets,

• staleness of compiled stylesheets isn't detected correctly,
• bundling of stylesheets can't be enabled/disabled depending on environment

[metaskills]

I've seen what you have been talking about and I'll read over this in detail when I finish up the asset helpers this weekend.

[stouset]

Great. Thanks!

[metaskills]

I'm not sure I want to do this one because it would require hacking how less.rb does it's @import. Cant we just leave this as is and not monkey patch LESS's @import?

[stouset]

Without Sprockets hooks, Less in Rails is flat-out broken.

Test this yourself. Run rake assets:precompile with your current stylesheets. Then change an @imported stylesheet and run it again. The cache-busting digest stamp will not have changed. Any browsers that already have the asset downloaded won't re-fetch it until it expires, which could potentially be years in the future.

The Sass project added hooks to handle this. Ask Less to do it too.

[metaskills]

@stouset As I was saying in the other ticket. Can please submit a patch with your ideas?

[stouset]

I would have already if I had the time. :)

[metaskills]

OK, then your likely to see this in a holding pattern for a week or more. So you have plenty of time to chip in and help out. I think I have us in a good spot with the current state and the testing system. FWIW, you may be able to get by using the depend_on directive.

[stouset]

That'll (barely) work for now (I have a lot of partials), but the core issue needs to be resolved before less-rails is ready for 3.1 apps in production.

For the time being, it might be worth filing a bug against less to add hooks for this. You know the internals better than I do, so you'd probably have a better notion of how best they could add support for it.

[stevenharman]

I just wanted to +1 the need for the hooks. There are workaround, like depends_on, but ultimately the hooks are needed for that It Just Works™ experience.

[cowboyd]

If you guys can point me to the sprockets hooks, I can have a crack at it. We are using less-rails in production, and are having this issue.

[metaskills]

@cowboyd I am not sure what @stouset was referring too when he said you had to just hook into sprockets. I can not find nothing in either sprockets or tilt that have to do with a standard way to do @import for CSS. In my opinion it is just out of the scope of the each project, but willing to learn where I might be wrong.

That said, sass-rails has behavior for @import which is heavily tied to Sass's @import and making it work with the pipeline. In my mind this get's complex because you would be in the area of over riding or hooking into LESS's @import and method chaining or injecting new behavior t make less-rails act like sass-rails, or more to the point, LESS work like SASS.

So maybe the place to look is the README on the sass-rails for the section called "Glob Imports" https://github.com/rails/sass-rails and the importer class https://github.com/rails/sass-rails/blob/master/lib/sass/rails/importer.rb

So I have no real direction at a quick glance, just places to go digging. Here is another, all the current issues on the less.js project with @import marked in them :) https://github.com/cloudhead/less.js/issues/search?q=%40import

[metaskills]

I should say above, I "can not find" any hooks.

[cowboyd]

Ok, I've had a look, and I think that it should actually be a pretty simple fix, but it will require monkey patching less.js at runtime.

If I understand correctly in this code, scope is an instance of Sprockets::Context which is unique to the asset being compiled. We can add any dependencies there, and sprockets will respect them.

To accomplish this, we will override tree.Import.prototype.toCSS() to add the path as a depdendency to the scope

A potential problem with this is that we'll need to inject the sprockets context as global variable into the environment, which could wreak havoc in multi-threaded environments. There might be a way to access the options passed into the parser from the the invidual AST nodes, which would be a godsend. I'll investigate that. Otherwise, we'll have to modify less.rb to allow for the creation of a unique JavaScript context for each asset.

[metaskills]

Yea, I thought about that potential problem in multi-threaded envs or apps with mountable engines by putting scope there. I use it in the template handlers. https://github.com/metaskills/less-rails/blob/master/lib/less/rails/helpers.rb#L94

Any thoughts in this would be welcome. I am not sure if my concern was valid or not, especially since or do not know how engines and that scope could be shared or not. Either way, me putting it in one place in the less code is very brittle.

[cowboyd]

Did I say simple fix? :) That was perhaps a bit too soon. The good news is that less does pass all options to the root toCSS() invocation to all child nodes.

To be clear, the only thing I'm proposing is to add the path passed to @import as a sprockets dependency. I would not change anything about the behavior of @import

[cowboyd]

Ok, I think we can rig both cases (helpers, and @import hooking) to options passed in at tree render time. That should cleanly avoid threading issues. e.g.

engine = parser.parse(data)
engine.to_css :sprockets => scope

I'll have a closer look at this tomorrow, but probably won't get anything working until next week.

[metaskills]

About ready to get back into this. Anything worth sharing before I do?

[cowboyd]

Turns out not to be such a simple thing. Turns out we need to patch in some non-trivial changes to less.js to get it to pass down options to all subnodes in the parse tree.

I still have that code lying around somewhere. I'll dig it out and have a look.

[joliss]

I can't contribute to the solution, but here is another reason why the @import probably needs to be integrated with Sprockets: Let's say you want to use mixins in your article.css.less file:

@import "twitter/bootstrap";
.article {
  .border-radius(15px);
}

But in comment.css.less you also need mixins:

@import "twitter/bootstrap";
.comment {
  .border-radius(15px);
}

Now if you want to compile everything into a single application.css (through require_tree .), the whole bootstrap library will be included twice, because Sprockets doesn't know about the duplicate @import.

[joliss]

(Using //= require "twitter/bootstrap" instead of @import will not work, of course, because then LESS doesn't know about the mixins.)

[joliss]

tl;dr: I was wrong; please ignore my last two comments.

It seems that I spoke too soon. Sass appears to have the same problem:

• @import causes duplicate imports.
• https://github.com/rails/sass-rails recommends against //= require, but I assume it's what most people are doing because of Rails's default application.css.

So I guess the preferred way would be to use //= require "twitter/bootstrap" in application.css.less, and @import "twitter/bootstrap/mixins"; wherever you need the mixins, so as to not re-include the entire framework. (Including the mixins is a noop until you actually use one of the mixins.)

Anyway, sorry for spamming. :-)

[metaskills]

OK, I am back into this one. I would love for some feedback on setting up a concrete understanding of what needs to be "fixed"

Let's assume I have some killer less framework nested away deep in my own vendor/assets or perhaps a gem and that this is hooked up correctly via less-rails load paths. For now the only thing this framework has is a single .mycolor function which is set to output color:green;. Here is the structure I have

File: app/assets/stylesheets/application.css

/*
 *= require application/base
 *= require_self
*/

File: app/assets/stylesheets/application/base.css.less

@import 'less_framework/base';

#container {
  .mycolor();
}

So assuming that I have config.assets.digest = true and I run the rake assets:precompile. I end up with the following two files with identical content of #container{color:green;}

• application-ec2842901042e01dc30f351daefbc5cf.css
• application.css

So now I go into my killer less framework and change the .mycolor() function to return red. This will mimic a simulated behind the scenes update of some framework gem. I destroy my public/assets directory and re-run the rake assets:precompile task. Both of these files below are generated with new identical content of #container{color:red;}

• application-ec1e498de79479c0b96ff4d08c83909b.css
• application.css

At this point I can easily see that the digest has changed. So initially I am not seeing how things are totally broken?

[stouset]

I replied by email, but it didn't seem to go through.

I destroy my public/assets directory...

This is broken. In no other asset engine is this required. Worse, if you forget this step, broken assets are pushed to your asset host with no prior warning.

In addition, asset-url and friends are broken. These statements go directly into the precompiled assets, and should return the relative path to assets, including digests. Without this, any background images or other assets can only point to the non-digested version, meaning client-side asset caching cannot be used.

[cowboyd]

I destroy my public/assets directory...

This is broken. In no other asset engine is this required. Worse, if you forget this step, broken assets are pushed to your asset host with no prior warning.

That is suboptimal, but I think calling it totally broken isn't completely fair. For the record the asset-* helpers do return relative paths with shas for us (we've been using this in production for several weeks); and, since we package each release and deploy it into a separate directory, there is never a chance of serving corrupted assets.

While it is a clear annoyance in development mode, it has performed flawlessly for us in production, which I think sets the bar for what is broken and what is not.

At the moment, my priority is refactor the less gem so that it can be used on jruby and rubinious, but I do have a personal desire to see this issue quashed. That said, the reason that this fix is non-trivial is that the less.js source needs to be modified to pass down context to each node as it is rendering the parse tree into CSS. It is inconsistent about which nodes receive context and which ones do not.

Unfortunately, @import nodes do not receive the full context. So, IIRC, the first question is, what is the changeset that is needed, and second, how to we get it into the JS source. Do we monkey patch it at runtime? Try to get it incorporated into the mainline source? maintain a fork?

[metaskills]

@stouset I'm having a hard time following you. Are you telling me that if I did not destroy my public/assets (i.e. run rake assets:clean) then I would not have gotten a new digest? If so that is incorrect. No matter which way I cut it, I always get a new digest. So without removing or running the approved blessed assets:clean task, I get these files. Note, the .gz versions where there in my last comment but just showing you them now. Seriously what exactly is broken?

• application-ec1e498de79479c0b96ff4d08c83909b.css
• application-ec1e498de79479c0b96ff4d08c83909b.css.gz
• application-ec2842901042e01dc30f351daefbc5cf.css
• application-ec2842901042e01dc30f351daefbc5cf.css.gz
• application.css

Re, your comment about asset-url, etc. Can you explain more too, are the tests failing for you? Or they totally written wrong? If I put this in my framework file.

.myimage() {
  background-image: image-url("less_framework/less.png");
}

Then I place that image in my frameworks assets/images directory. Then I change my own base.css.less like so.

#container {
  .mycolor();
  .myimage();
}

And recompile I get these in my assets directory along with this content #container{color:red;background-image:url(/assets/less_framework/less.png);} in each of the application.css files. Please details what is wrong.

• less_framework/less-ef9b99c1df7d13853cb626bdf3f1f269.png
• less_framework/less.png

[metaskills]

Thanks @cowboyd I recently saw that less.rb was not jRuby happy by a comment from @drnic on another gem that uses less-rails. jRuby is not a priority of mine, nor any interest, nor should we muddy the waters on this ticket. To that end, this ticket was about (1) wanting to @import .css.less extensions and (2) saying that @import is totally borked if files it imported were changed. Thru this discussion, it was mentioned about digests in asset url helpers. I have opened up ticket #9 for that so it is easy to track and not have a single ticket be a throw crap at the wall scenario.

So, back on my understanding of the two points of this ticket. I think (1) is either a misunderstanding of how to use Less with the asset pipeline and where we should use sprockets or less's @import. More to the point how to make Less more like Sass. I see no value to using @import for a foo.css.less file that would need to go thru the whole sprockets rendering context again. If so, can someone point out why we need to worry about it because @stouset's scenario for the (2) concern has thus far not been true.

[metaskills]

So in short, can someone show me some code that demonstrates what your expectation or problem is?

[stouset]

That is suboptimal, but I think calling it totally broken isn't completely fair. For the record the asset-* helpers do return relative paths with shas for us (we've been using this in production for several weeks); and, since we package each release and deploy it into a separate directory, there is never a chance of serving corrupted assets.

While it is a clear annoyance in development mode, it has performed flawlessly for us in production, which I think sets the bar for what is broken and what is not.

The fact that you've accidentally worked around the problem doesn't mean it's not broken. :)

Here are the bits of less-rails that are not working correctly, and a comparison with the Sass plugin. First, the players:

// $ cat application.css
//= require 'application-scss'
//= require 'application-less'

// $ cat application-scss.css.scss
@import 'foo';
#container { @include foo; }

// $ cat _foo.css.scss
@mixin foo { color: white; }

// $ cat application-less.css.less 
@import 'bar';
#container { .bar(); }

// $ cat _bar.css.less 
.bar { color: white; }

@import doesn't work out of the box with sprockets' file naming conventions

$ curl localhost:3000/assets/application.css
Less::ParseError: file 'bar.less' wasn't found.

Let's explicitly ask for a partial.

// $ cat application-less.css.less 
@import '_bar';
#container { .bar(); }

$ curl localhost:3000/assets/application.css
Less::ParseError: file '_bar.less' wasn't found.

No joy. But if we import it explicitly?

// $ cat application-less.css.less 
@import '_bar.css.less';
#container { .bar(); }

$ curl localhost:3000/assets/application.css
/* line 2, /Users/stouset/less-test/app/assets/stylesheets/application-scss.css.scss */
#container {
  color: white;
}
.bar {
  color: white;
}
#container {
  color: white;
}

Success!

@import doesn't work with Sprockets' asset freshness checking

Let's go ahead and make some changes to our assets.

// $ cat _foo.css.scss 
@mixin foo { color: black; }

// $ cat _bar.css.less 
.bar { color: black; }

$ curl localhost:3000/assets/application.css
/* line 2, /Users/stouset/less-test/app/assets/stylesheets/application-scss.css.scss */
#container {
  color: black;
}
.bar {
  color: white;
}
#container {
  color: white;
}

Wait, what? Why are my less styles unchanged? Even restarting the Rails server doesn't fix anything. You have to run rake tmp:cache:clear or add blank padding lines to the outermost Less to trigger a refresh.

Note: this doesn't seem to be broken in production (rake assets:precompile will always use the latest) as of Rails 3.1, however, rake assets:precompile is broken for existing Rails 3.0 apps. I don't remember the exact details, but I believe it replaces the contents of existing stamped-assets without changing the stamp (which is a huge showstopper for anyone serving cached assets [which damn well ought to be everyone]).

[stouset]

To preempt the reply about using require instead of @import:

$ cat application-less.css.less 
//= require '_bar'
#container { .bar(); }

$ curl localhost:3000/assets/application.css
Less::ParseError: .bar is undefined

Basically, Sprockets evaluates each require'd file in isolation before attaching them all together.

[metaskills]

@stouset This ticket is not for digests, please go to issue #9.

[stouset]

I was listing the entire set of problems caused by not integrating with Sprockets (which was the original focus of this issue). The first two problems have nothing to do with asset digests.

[metaskills]

Let me recap for @stouset. You think we should monkey patch Less' @import for the following reasons.

Run rake assets:precompile with your current stylesheets.
Then change an @imported stylesheet and run it again.
The cache-busting digest stamp will not have changed.

This is incorrect, if the imported stylesheet affects the final outcome, it will change the digest as I have shown. Your statement you've accidentally worked around the problem doesn't mean it's not broken thought technically true, you could take a step back and say something like, oh sorry, I meant in development things were not working and I assumed that precompile would not work, etc.

@import doesn't work out of the box with sprockets' file naming conventions

And maybe it should not. If that is the case, this will become a documentation challenge to educate people why we have chosen not to do this along with examples of how to use Less.js in the asset pipeline. So in your example code, just rename _bar.css.less to bar.less.

Let me restate, I am not going to do anything simply for the sake of Less to Sass parity. There are some things that I think are logical to do, like generators, asset helpers, etc. But just by saying that Sass (sass-rails) does something is not going to accomplish anything. You will have to be make a clear case with a lot less hyperbole than your previous comments.

@import doesn't work with Sprockets' asset freshness checking

Now that could be annoying. It is also the first time you have articulated the problem too! I would hate myself to be working on my killer new Less framework along side my awesome app, then change the way a function does something and have to clear the cache all the time. So far this is the only worthwhile reason I have seen to monkey patch @import. I will look into this specifically and followup.

[stouset]

This is incorrect, if the imported stylesheet affects the final outcome, it will change the digest as I have shown.

From my above post:

Note: this doesn't seem to be broken in production (rake assets:precompile will always use the latest) as of Rails 3.1, however, rake assets:precompile is broken for existing Rails 3.0 apps. I don't remember the exact details, but I believe it replaces the contents of existing stamped-assets without changing the stamp (which is a huge showstopper for anyone serving cached assets [which damn well ought to be everyone]).

However, I fucked up the version numbers. Rails 3.1.1 does not exhibit this problem, Rails 3.1.0 does.

Now that could be annoying. It is also the first time you have articulated the problem too!

From my very first comment:

The biggest issue is that Sprockets uses its internal representation of the import hierarchy to determine staleness. This means that updated Less partials won't take effect until you make a change to the topmost stylesheet.

[stouset]

I've updated my previous comment to remove the asset fingerprinting complaint. Per issue #9, I am an idiot and was missing the fact that only fingerprinted assets use fingerprints for the assets they include.

[metaskills]

On a lighter side, I wonder how many github notifications this poor guy has gotten throughout this discussion :)
https://github.com/import

[metaskills]

Offline gist: Failed ImportProcessor
https://gist.github.com/1368396