when a url path has query parameters, the url rewriting does not add the digest (case bootstrap icons) about propshaft

[koenhandekyn]

this issue leads to questions like here. the answer only works with sprockets though because sprockets also serves the assets without the digest.

[brenogazzola]

Hello @koenhandekyn.

Could you please confirm which version of Propshaft you are using, and how bootstrap is being integrated into the project (cssbundling-gem, etc.)? Also, could you also use assets:reveal:full task and add to your response the full path of the file (including any digests that were added)?

The bug seems weird to me, because we have a test case for that exact scenario, which originated from this very problem caused by bootstrap:

test "fingerprint" do
  compiled = compile_asset_with_content(%({ background: url('/file.jpg?30af91bf14e37666a085fb8a161ff36d'); }))
  assert_match(/{ background: url\("\/assets\/file-[a-z0-9]{40}.jpg\?30af91bf14e37666a085fb8a161ff36d"\); }/, compiled)
end

[koenhandekyn]

initially we used just the latest greatest propshaft released but then we switched to 'main/master' from github just to be sure, see below info for versions and the relevant details for reveal. cssbundling-gem with initialisation mode bootstrap indeed.

BUT I think i've found some more details about the issue !!! see further

ruby 3.1.2

GIT
  remote: https://github.com/rails/propshaft.git
  revision: 467f2a13cfc3d30a3366408c8f98c2deb4efc741
  specs:
    propshaft (0.6.4)
      actionpack (>= 7.0.0)
      activesupport (>= 7.0.0)
      rack
      railties (>= 7.0.0)

☻  bundle exec rake assets:reveal:full | grep fonts                                                                                                                            /Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.ttf
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.min.css
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.woff
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.eot
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.otf
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/README
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.css
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.svg
/Users/koenhandekyn/Work/woodworks/app/assets/fonts/Code39Azalea.html
/Users/koenhandekyn/Work/woodworks/node_modules/bootstrap-icons/font/fonts/bootstrap-icons.woff2
/Users/koenhandekyn/Work/woodworks/node_modules/bootstrap-icons/font/fonts/bootstrap-icons.woff

☻  bundle exec rake assets:reveal | grep fonts
fonts/bootstrap-icons.woff2
fonts/bootstrap-icons.woff

this is the situation with this in assets.rb to extend the load path

Rails.application.config.assets.paths << Rails.root.join("node_modules/bootstrap-icons/font")

HOWEVER ( ROOT CAUSE FOR THE ISSUE? )

as soon as we add node_modules itself also to the load_path, then the fonts get exposed ONLY from the node_modules path with the bootstrap-icons/font as prefix and not directly without the prefix, as would be needed.

Rails.application.config.assets.paths << Rails.root.join("node_modules")
Rails.application.config.assets.paths << Rails.root.join("node_modules/bootstrap-icons/font")

this gives below and mind the prefix path appearing (and only those)

☻  bundle exec rake assets:reveal | grep fonts                                                                                                                                 
bootstrap-icons/icons/fonts.svg
bootstrap-icons/font/fonts/bootstrap-icons.woff2
bootstrap-icons/font/fonts/bootstrap-icons.woff

this behaviour seems to be different from sprockets, and it might make sense to revise this.

[brenogazzola]

Hmmm... I don't quite remember what is sprockets behavior. For the assets folder in app, vendors and lib Propshaft does unnest the first level of folders, since it's rails convention that you won't have any files in there, only folders. But for folders you add yourself, Propshaft will not do that and will simply copy the structure as is.

Looking at your full assets it seems you are missing the bootstrap-icons.css file? I see that in the node modules folder it is there as .scss. How is it being transpiled to css? The correct procedure here is to have the transpiler place the css file in assets/builds folder so that Propshaft can pass it through the CSS compiler and adjust its url attributes to include the assets' digests. If you are placing it directly in public/assets that might be the problem.

[koenhandekyn]

fyi, the css file is converted from scss to css by esbuild that puts it in assets/build/... (should be pretty standard)

my conclusion is that propshaft should expose assets twice if they appear twice with different prefix paths in the asset paths, currently it exposes just one.

concretely imagine with have this file structure

node_modules/bootstrap-icons/font/fonts/somefont.woff

and we have these asset paths

Rails.application.config.assets.paths << Rails.root.join("node_modules")
Rails.application.config.assets.paths << Rails.root.join("node_modules/bootstrap-icons/font")

then propshaft should probably expose 2 variants

bootstrap-icons/font/fonts/somefont.woff # from the first asset path
fonts/somefont.woff # from the second asset path 

currently it seems to always expose only "bootstrap-icons/font/fonts/somefont.woff" once the first asset path (just node_modules) is added, independent of the ordering.

[brenogazzola]

Propshaft intentionally dedups the paths, keeping only the ancestor of the two for performance reasons.

[brenogazzola]

Also, now that I think about it, adding the entire node modules folder is probably a misconfiguration, as that will cause Propshaft to copy the entire folder to public/assets. Which you probably don’t want or need.

Propshaft does not have a manifest file like sprockets, so it really copies everything in the load path.

[koenhandekyn]

then maybe this could be made more explicit in the documentation within the migrating from sprockets section? adding node_modules as the asset path is probably a pattern that might be more common than estimated? but I fully agree that in a 'clean' configuration, exposing all of node_modules is probably in any case not something which is desirable.

it might even make sense to throw an error for the user when one asset path is a subdirectory of another or more generally whenever deduplication takes place, as this likely is leading to unexpected behaviour ?

[koenhandekyn]

i'm closing this issue as the issue is something unrelated and seemingly a design choice. it's up to the owners to decide if they want to add some extra documentation and or warnings when propshaft is deduplicating.

thanks for the quick responses

[brenogazzola]

Yup, lot's of stuff to do before version 1.0. We are still figuring out where Propshaft is breaking or causing confusion. At the time I programmed the deduplication it made sense for my specific use case, but it seems to have caused some confusion in your case, so we might need to change that like you suggested.

I'll write this down as a to-do so we can discuss this.

Thanks for testing Propshaft and reporting the problems! 🙇‍♂️