Include files coming from engines about webpacker

[dhh]

Yeah, we need a way to add plugin paths to the node load path, just like asset pipeline does it. Not quite sure yet how. But you can see how things are setup to use vendor/node_modules and app/javascript in the config files.

[mengqing]

Wondering if there is any updates to this issue. We're currently facing this particular issue and is trying to find out if there are any viable solutions

[tf]

One of the main issues, from my point of view, is how to handle npm dependencies when extracting JavaScript into engines. Just adding engine paths to the load path won't ensure dependencies are installed. Including engine package.json files in the main app's yarn install can lead to problems with conflicts between bundler's and yarn's dependency resolution.

[dhh]

@guilleiguaran Is it possible to use the erb loader on the webpack config? Because if so, we could scan all the plugins for a specific directory to add to webpack's load path and just insert it it directly.

[jcoyne]

It seems like we'd want to set resolve.modulesDirectories, but I don't see a CLI option for setting that.

[jcoyne]

Would a PR like this be helpful? webpack/webpack@master...jcoyne:resolve-modules-directories

[renchap]

I am working on https://github.com/renchap/webpacker-react and this is my biggest issue at the moment.
Currently (push to the repo coming soon) I created a webpacker-react NPM package and you need to manually include it into your package.json in addition to adding the gem to your project, and it would be great to be able to avoid this step.

[josepjaume]

I'm also having this issue! It should be possible to depend on a package.json from the

[josepjaume]

🤓😂... gem the same way we do with gemspecs, right?

[tf]

Well, yes, but that would leave Yarn with dependency choices already made by Bundler, which in some cases might not be optimal. In particular, to really get to the most up to date versions, users might have to first bundle update and then yarn update. We would probably also need to apply some magic to the application's package.json to insert the (transitive) engine dependencies. I'm not sure all of this will make for a very intuitive workflow.

[dhh]

I don't think we have a good path for doing this given that such inclusion wouldn't be able to resolve any dependencies. I don't see a way out of having NPM packages for the plugins. That's how we've gone with Action Cable and Active Storage for now. I'm closing this lest someone else comes up with an alternative idea alongside a proof-of-concept PR that deals with the dependency issue.