Naming inline modules about loader

[matthewp]

This would not be possible under the current spec due to there not being a concept of module names any more. Everything is keyed by url so virtual modules are an awkward fit now (hopefully this decision will be reconsidered, read #28). Add this to the pile of benefits of module ids over urls.

Also, neither System.define nor System.module have counterparts in the current spec.

Surely it makes the most sense, as @dherman pointed out at jorendorff/js-loaders#83 (comment), to do the following:

1. Associate inline modules without a name attribute with System.module.
2. Associate inline modules with a name attribute with System.define, but only if a path is not set for that name. (This should have the added advantage of showing up within the sources panel as something other than <Anonymous Module>.)
3. For any modules with a src, use the address-normalized path as usual. And for any modules with a name, use the name-normalized path as long as it does not yet exist. It should also be possible to use both the name and the src paths if both are provided.

I can't imagine any conflicts occurring with this system.

[caridy]

@matthewp that's not accurate, you can have any arbitrary entry in the registry.

Loader.install() will allow you to define new entries in the registry: https://whatwg.github.io/loader/#reflect-loader-install

The normalization process (to a url), including the sites() configuration will happen if there is no entry available in the registry for an arbitrary values in the module specifier.

[matthewp]

@caridy Interesting, install() doesn't specify a return value, this means it could be a Promise. This differs from the old System.set / System.get.

[matthewp]

@timbur This spec is quite a bit different from the one referenced in that thread, we need a Loader.define and Loader.module first. I'm not sure what the authors' opinion is on .define, I remember there was some problems with the old version. .module will almost definitely be needed to support <module>. Maybe a separate issue to track that?

In my opinion, es6-module-loader and systemjs are handling everything perfectly and intuitively, so it would make sense to model anything missing from this spec after their functionality. Plus, those projects are designed to match this spec, so it's a bit of a win-win for everyone.

[caridy]

@timbur systemjs should not dictate what we do here, it is the other way around. We are considering those ideas that we have tried, and play around with in systemjs, but ultimately we have to be pragmatic.

@matthewp I think you're confused. install() does not fetch deps since you have to provide a [[Module]] instance, which produces a new entry in the registry with [[State]]: "ready", there is nothing to return.

I didn't mean to imply that SystemJS should dictate anything here. I was only saying that there is some extra functionality present within SystemJS which makes a lot of sense, and this functionality exists because of practical, real-world necessity/usefulness. I'll provide an example of said necessity shortly (in addition to the example I provided above).

[guybedford]

This spec has a loader.provide which actually gives more flexible behaviour than the System.define, System.module. See https://github.com/ModuleLoader/es6-module-loader/blob/1.0/src/loader.js#L429 for how it works.

@timbur note that name is deprecated in this spec, and modules are only src based.

[matthewp]

@caridy I might be confused, let's break this down:

The normalization process (to a url), including the sites() configuration will happen if there is no entry available in the registry for an arbitrary values in the module specifier.

This normalization process is (potentially) asynchronous, if it goes through the resolve hook. If that's the case it should return something.

[caridy]

@matthewp the normalization process only happens when importing a module (or a dep), not when installing a new entry in the registry.

[matthewp]

Ok, then I did misunderstand you. So this mean keys are really just arbitrary strings, I can do:

System.install("@@foobar", { default: function() { return "bar"; } });

And anyone importing that will get the correct module, cool.

@guybedford Can that same flexible behavior be achieved directly through System? Or is it currently limited to methods within loader?

As for the real-world necessity, it makes no sense to render inline modules inaccessible. They're modules, too, right? It should be possible to import them. It also doesn't make sense to give them the src attribute since the source is contained within itself. The only logical conclusion, in my opinion, is to use name so that said inline modules can be imported by their names.

The way I see it, from a simple point of view and for predictability's sake, internally, when paths are set and modules are registered, all that matters in the end is the corresponding key. A few simple rules and everything gets insanely simple:

1. Is there a path match? If so, use it and adjust the key to its full path name.
2. Are there any identifiers that would indicate it's a file (src, has a file extension, etc.)? If so, use it and adjust the key to its full path name.
3. A single word with no extension? Don't do anything fancy. Don't change it. Just use it. At this point in these 3 steps, it's safe to assume we're looking for an inline module that was registered using <module name="..."> or explicitly defined some other way. And if it isn't already registered, hit the server without appending .js to it.

Anything beyond that would be convoluted, adding unnecessary complexity and wasting everyone's time. Keep it as simple and predictable as that and the only conceivable way developers could screw up is if they purposely shoot themselves in the foot.

[MajorBreakfast]

I like the simplicity of having just src.

What about this:

<module src="hello.js">
export default 'I am a module'
</module>

[matthewp]

@MajorBreakfast that means src has a different meaning depending on whether there is inner text or not. Would also differ from every other use of src attributes in html.

[MajorBreakfast]

@matthewp Hmm, yes. Maybe a <script> tag + System.install() (or provide, not sure) comes close enough already. Likely, there even shouldn't be a more convenient way to do this.

[caridy]

@timbur no. we have discarded that a while ago, and we've settled on the idea that inline modules will probably be focus on initialization (just like inline scripts are today), and therefore there is really no need to export things into the runtime from them, instead they just import other modules, and carry on with some initialization routines. I'm curious about what sort of inline script code you have today that modifies the runtime to extend the available functionality.

[matthewp]

@guybedford I missed your comment but you're right that .provide is a better .define. Don't we still need a key-less form, though? By default a <module> shouldn't be part of the registry.

@caridy My issue with that line of thinking is that if developers want to do something, they'll figure out a way to do it. It's always better to provide an elegant, predictable solution for people to use, rather than imposing arbitrary restrictions that will lead developers to create ugly hacks/workarounds. And for this particular case I'm 100% certain that will happen.

Consider this scenario: Users load some module specific to their profile. We'll refer to it as a duck. Every user has their own duck with custom functionality that could not be easily defined in any form other than JavaScript. It's initialized upon loading the page, and many other modules want to use that particular duck, expecting a quack method that is potentially custom (different) for each user.

Suppose we have my duck at timbur/duck.js:

export function quack () {
    // let's pretend this is all kinds of complex logic
    console.log('quack!');
}

And we have your duck at caridy/duck.js:

export function quack () {
    // let's pretend this is all kinds of complex logic
    alert('quack!');
}

And when I load the page, the HTML generated for me looks something like this:

<module>
    import duck from 'timbur/duck.js';
    duck.quack();
</module>

And yours of course looks something like this:

<module>
    import duck from 'caridy/duck.js';
    duck.quack();
</module>

And later on, perhaps some other module wants to make the user's particular duck quack, we'll call it goose.js. There are of course many different ways to make this happen, so let's consider a few.

Without the ability to name (and register) the <module> above, it's really not even a module. It's just a script. So, as a script, if we want to maintain some reference to the user's duck, the script itself will need to assign the duck to some global variable.

<script>
    import duck from 'caridy/duck.js';
    duck.quack();
    window.main = {duck: duck};
</script>

Or for the sake of consistency and "elegance" (if you could call it that), one might pull some other module designed specifically for storing this information:

<script>
    import store from 'store.js';
    import duck from 'caridy/duck.js';
    duck.quack();
    store.set('duck', duck);
</script>

And then for our goose.js module to use the user's particular duck:

window.main.duck.quack();

Or:

import store from 'store.js';
store.get('duck').quack();

Now let's be honest. The above solutions are ugly and they go against the grain of the module system.

Of course, all of this so far might seem like a contrived example, and most people might find this particular usage an edge case. But again, it's never a good idea to impose restrictions on what should or shouldn't be possible only because it isn't some scenario that is easy to imagine. People are obviously already aware of these possibilities, so why limit them? It's much better to provide an elegant solution at the core.

And finally, let's consider a solution that I would probably use if naming an inline module was not built into the core of <module>'s functionality. I realize this solution relies on System.define, but let's not be pedantic. The analogy remains.

HTML snippet:

<module name="main">
    import duck from 'timbur/duck.js';
    duck.quack();
    export { duck };
</module>
<script src="register-inline-modules.js"></script>

register-inline-modules.js:

var modules = document.getElementsByTagName('module');

for (var i = 0; i < modules.length; i++) {

    var module = modules[i];

    if (module.hasAttribute('name')) {
        var name = module.getAttribute('name');
        var source = module.innerHTML.substr(1);
        System.define(name, source);
    }

}

Then goose.js would very predictably look like this:

import main from 'main';
main.duck.quack();

That is way simpler and follows the module paradigm much more consistently. But we aren't finished yet.

Last but certainly not least, consider that the name attribute is supported. The following is all we need. No global pollution. No modules designed specifically to remember duck. No unnecessary extra scripts. It's intuitive and succinct. Just use the registry.

HTML snippet:

<module name="main">
    import duck from 'timbur/duck.js';
    duck.quack();
    export { duck };
</module>

goose.js:

import main from 'main';
main.duck.quack();

It's entirely possible that I'm being stupid and completely overlooking an easier way to achieve this. And if that is the case, please let me know and I will gladly shut up about this particular example. Either way, I think there are more than enough use cases out there that we haven't thought of which will resemble the above functionality, so why not make it elegant?

[MajorBreakfast]

@timbur I think the idea is to have a main.js file (Directly or included in a bundle, e.g. the way SystemJS does it). The module tag code then simply imports that module.

It's always better to provide an elegant, predictable solution for people to use [...]

Not if the solution can be considered a dead end. I think it's good to separate JavaScript from HTML.

But it isn't a dead end. The key point is that it should be possible to elegantly register dynamic modules under some common name without resorting to extra scripts. Yes, in production, all static modules will be minified and bundled. But it would be stupid to create separate bundles for each user due to some dynamic module reference. The dynamic module would obviously automatically be excluded from the bundle, and the most efficient way for modules within the bundle to use it is to have its reference exported upon initialization. I don't know how I can make it any clearer, and I've yet to see a reasonable argument against this functionality, nor a more elegant solution to the problem.

[johnjbarton]

Can the same effect be created with

<script> System.sites = {main: 'timbur/main.js'}; </script>

?
Like Tim's example, this would be a per user specific page script. Like
Tim's example, the effect is to cause goose.js to see a different main
module dynamically, when it runs import. Like Tim's example it makes
development more challenging because the module is dynamic.

Unlike Tim's example it does not require a with a name.

(To be sure, IMO we don't need a tag without a name either. A

<script> tag with System calls are clearer and provide better developer ergonomics in realistic code.) jjb On Fri, Apr 3, 2015 at 5:08 AM, Tim Burgess [email protected] wrote: > But it isn't a dead end. The key point is that it should be possible to > elegantly register dynamic modules under some common name without resorting > to extra scripts. Yes, in production, all static modules will be minified > and bundled. But it would be stupid to create separate bundles for each > user due to some dynamic module reference. The dynamic module would > obviously automatically be excluded from the bundle, and the most efficient > way for modules within the bundle to use it is to have its reference > exported upon initialization. I don't know how I can make it any clearer, > and I've yet to see a reasonable argument against this functionality along, > nor a more elegant solution to the problem. > > — > Reply to this email directly or view it on GitHub > https://github.com//issues/42#issuecomment-89268037.

That's a pretty solid suggestion @johnjbarton, albeit still not nearly as elegant and intuitive as it could be. And I actually agree that a <script> tag with System calls is more than sufficient. However, I personally feel that web development in the future will take place almost entirely within the browser itself. I'll explain my reasoning and what it has to do with modules.

We're already seeing a ton of movement towards minimizing the feedback loop with development - e.g., webpack hot module replacement, react-hot-loader, workspaces within dev tools that automatically save changes to CSS within the inspector, etc., etc. So how can we get the feedback loop to an absolute minimum? By having all of our development tools directly in the browser. And of course, modules are undoubtedly the future, so we need a really solid module system that works seamlessly with this paradigm. It needs to be effortless to develop modules and make them work together, and even inline modules should be identifiable. It's important to know what's what.

With that said, let's compare what an in-browser development environment would look like without named modules. Also consider that there might end up being more than a handful of anonymous modules in the development environment, for whatever purposes.

And this is what it looks like with the named inline module. It's more intuitive, much easier to decipher.

[caridy]

@timbur are you talking about bucket testing? jejejeje

Anyway, @johnjbarton is righty, loader.sites() is really the way to go for those use-cases.

[MajorBreakfast]

@timbur Each HTML file should only contain one module tag. The named file that shows up in the dev tools cannot be edited and saved because it exists on disk inside the index.html. Why not use a main.js file?

@caridy I'm not talking about bucket testing at all, but that's a perfect example of one of the many possibilities that would be easier (or more elegant, at least) if naming inline modules is supported. Can you provide an example using loader.sites()?

@MajorBreakfast You're thinking inside the box. In an ideal in-browser development workflow, there are all kinds of possibilities that could potentially result in multiple html files within the sources panel and thus, multiple inline modules. As for using a main.js file, the code would be different for every user, so it makes much more sense to be an inline script.

[caridy]

@timbur and co, our goal is not to provide sugar for every use case but for the most common ones, and make sure that the rest can be implemented in userland. This use case can be implemented in userland, no doubt about it, and We haven't seen this one very often when people embrace the ES module philosophy, It is really more like a side effect of trying to make modules to behave like scripts, which is fine but ... it is not really something we will promote or try to provide sugar for. If you really want to continue this conversation, you should demontrate that the currect spec does not allow you to do it on userland, or came up with a real very common use case that will requiere named inline modules. Makes sense?

Of course just about anything can be achieved in userland. The point I've made is that including this functionality within the spec would only result in one or two lines extra of code at core of <module>, and these one or two lines would result in more than enough predicted benefits to warrant its inclusion, along with possibly many many more unpredicted benefits and no downsides whatsoever.

[probins]

came up with a real very common use case that will requiere named inline modules

one that I've just been investigating is HTML Imports. Currently most existing web components that I've seen that use HTML Import have an inline template plus an inline script to deal with the template. In some cases, that script will just be an initialisation of some sort, but in others the component can be interactive and expose some sort of API - getThis, setThat. If these scripts are converted to modules, either there will have to be a rule that they should always link to an outside file (which might make sense from CSP point of view), or they will have to be named, so other modules can import them.

[probins]

a simple example of this is Polymer. https://github.com/Polymer/polymer/blob/master/dist/polymer.html is the main component which runs a script to load the Polymer class function which all Polymer components use. This is currently a global. If you change this structure to modules, then all components have to import the Polymer module. To do that, it has to be named.

[MajorBreakfast]

AFAIK custom elements are global in nature. The API can live on the element.

It's essentially:
HTML files as bundles <-> Extended loader that supports JS files as bundles (-> SystemJS)

Edit: Only document.registerElement() part of creating a custom element is global. The actual element class doesn't need to be a JS global. Anyway, I'm not 100% sold on the HTML import part. It's like the JS loader mechanism, but less ambitious.

[guybedford]

One could imagine that script being upgraded to a module:

<script type="module" src="polymer.js"></script>

Where 'src' above is an unnormalized module name (it may be called 'key' or something else though).

There is no need to name the module in the above, it is loaded anonymously and stored in the registry like any normal module load. Defining how modules affect rendering is the next question there though.

[probins]

@guybedford You're saying that <module src="x.js"> would be handled differently from <script src="x.js">? See the comments from Apr 2. And what about inline modules in HTML Imports? If they export values, they need to have an identifier.

The basic issue here is that you are not importing a module file with a url, you are importing a document fragment which contains one or more module definitions which may need to be loaded in the registry, System.define() in the old spec, loader.install() in the new.

[probins]

@MajorBreakfast HTML imports don't have to be custom elements. They could be a form or a bunch of divs or whatever grouping of html elements you like.

[probins]

Having raised this issue, I'm not sure there is actually a problem. :-) The simple solution for the case where a component has to export values is to split the code in two: the <module>, whether inline or external, does the initial setting up, and if there's a need for exporting, then that code can be put in a separate js file which can be imported in the usual way. A disadvantage of this would be that the separate js file would be detached from the HTML Import and not necessarily linked to from it. But this is also the case with the main document: you can't tell from the html which modules are loaded (form part of the app). In the Polymer case, I don't think you need a Polymer component with a <module>, just a standard module for the Polymer class that can be imported by Polymer components.

In summary, a web app is an html document that can import stylesheets and components, and can contain an anonymous <module> to bootstrap the app. A component is an html document fragment that can ... to bootstrap the component.

[caridy]

correct, none of these examples really require an inline module to be named, lets keep it that way for now. We can always revisit this if needed :)