In particular, given a package name map like the following:

{
  "packages": {
    "https://example.com/foo": { "main": "node_modules/src/foo.js" },
  }
}

and a JS file like

import "https://example.com/foo";

should the result import the file from https://example.com/foo, or from node_modules/src/foo.js?

Stated in spec terms, should the package name map resolution come before or after step 1 of resolve a module specifier?

I really like the extra power of allowing overriding of URLs. It does depart from the literal sense of "package name map" though, and might imply a renaming of this project to something more like "module resolution map" or similar.

First off great to see explorations on this... some feedback.

Consider the following case...

{
  "path_prefix": "/node_modules",
  "packages": {
    "index.js": { "main": "src/index.js" },
  }
}

In an application that also has an index.js at it's root. What does import * from 'index.js' load? That is, how does this deal with naming collisions?

There's a danger here in recreating the hell that is Java classpaths.

Another approach to this could be to take a more generic http/http2 centric approach and allow an origin to declare client-side aliases for various resources. These would amount to a client side 302 or 303 response.

For instance:

HTTP/1.1 200 OK
Link: </node_modules/lodash.js>; rel="alias"; anchor="lodash"
Content-Type: application-js

import * from "lodash"

Does Node.js need additional configuration? can it be in package.json?

If I understand well, this proposal may cover webpack's custom resolvers and aliases? and the possibility to have internal import 'bindings' like:

I've often things like

	resolve: {
		extensions: ['.js', '.json'],
		modules: ['src', 'node_modules'],
		alias: {
			assets: `${__dirname}/assets/`,
		},
	},

that would benefit being standardized. Since it's needed by test engines and linters also

Currently it seems like the matching in the reference implementation is done sequentially. I do have some concerns here from both a specificity and performance perspective as with hundreds or thousands of packages this could incur overhead for each package resolution (~O(rn^2) where n is the number of packages and r is the number of specifiers for a package - 500 packages loaded means 500 lookups against 500 sequential items, where a given package might be itself imported 10s of times giving say 2.5 millions iteration checks just for package names before looking into scopes).

Specificity-based matching which could be preferable for cases like:

{
  packages: {
    "jquery": { "main": "index.js" },
    "jquery/x": { "main": "subpackage-main.js" }
  }
}

where one might want subpackages to work out naturally, without having to construct some type of special scope precedence trick to get the right matching.

Typically I use dictionary lookups here, specifying these matches in terms of specificity (longest match wins), and then walk down the path segments with dictionary lookups.

So to check a specifier a/b/c I would first check a/b/c in the dictionary lookup, then a/b in a dictionary lookup, finally followed by a.

This type of matching process discussed could apply equally to scopes and package names.

As noted in one of the examples, we could allow shortening of { main: "moment.js" } to just "moment.js". Then you could write package map files like:

{
  "path_prefix": "/node_modules/",
  "packages": {
    "moment": "moment.js",
    "html-to-text": "index.js",
    "redux": "lib/index.js"
  }
}

which is appealing for its simplicity. (Note how we still assume there's a path segment/folder corresponding to the package name, so e.g. redux is located at /node_modules/redux/lib/index.js. That follows from the default value for "path".)

This slightly complicates the data model, as it would be the first instance of a union type in our structure. But it doesn't seem like that big of a deal. Should we do it?

Preferably:

• Using packages that exist on npm
• Using packages that would be used on the frontend
• Not involving lodash, since we use that as our example of a package's name (lodash-es) and its specifier (lodash) not matching
• Non-dev dependencies

This would replace our current fake example.

I'm wondering, how is this repository licensed? I noticed that there are contributions here from both Googlers and non-Googlers (e.g., @guybedford), so relicensing later might not be trivial. I don't see a license file checked in. For a standards project like this, I imagine that the license would cover both copyright and royalty-free patent policy. Note that working within a standards body would solve the license issue.

I spotted the following line in the proposal:

Notice also how the full URL path of the nested lodash package was composed: roughly, top path_prefix + scope name + scope's path_prefix + package's path`. This design minimizes repetition.

Let's say I have the following package-name-maps:

{
  "path_prefix": "https://foo.bar",
  "packages": {
    "A": { "path": "A@1", "main": "index.js" },
    "B": { "path": "B@1", "main": "index.js" }
  },
  "scopes": {
    "A": {
      "packages": {
        "C": { "path": "C@1", "main": "index.js" }
      }
    },
    "B": {
      "packages": {
        "C": { "path": "C@2", "main": "index.js" }
      }
    }
  }
}

According to the line I quoted, it means that:

• Resolving A from the toplevel will call https://foo.bar/A@1/index.js
• Resolving B from the toplevel will call https://foo.bar/B@1/index.js
• Resolving C from A will call https://foo.bar/A/C@1/index.js
• Resolving C from B will call https://foo.bar/B/C@2/index.js

I feel like the first two resolutions make sense, but the two others don't. Is automatically adding the scope name to the URL (with no way to prevent this short of maybe using ../ as path_prefix, which is unintuitive and whose behavior isn't clearly defined) a good idea?

Since this file will end up being automatically generated in most cases, it seems a bit premature to optimize for less repetition.

What would be the exact form of the path, package name and scope name?

Enforcing all of these as bare names might roughly do the job, but does it cover it sufficiently?

Further, if I have one invalid package name in "packages", or one invalid scope name in "scopes", does this validate on startup? How does it throw / warn?

Do all other validations happen during resolution itself? If so, do their validations throw as the resolution error?

I don't see the benefit of:

• path_prefix (Maybe for size reasons? Doesn't gzip also eliminate this?)
• Separation of path and main (Maybe for defaulting path?)

And instead propose to omit these things. I also propose to make scopes a first-class-citizen which leads to the following format. (Assuming this format will be created by a tool and not by human.)

{
  "/": { // scope (origin of the request)
    // alias for a module
    "moment": "/node_modules/moment/src/moment.js",
    "lodash": "/node_modules/lodash-es/lodash.js",
    // alias for a inner request
    "lodash/pluck": "/node_modules/lodash-es/lib/pluck.js"
    // alias for all inner requests
    "lodash/*": "/node_modules/lodash-es/lib/*",
    // alias for full urls: i. e. for extensions
    "/my/app/file": "/my/app/file.js"
  },
  "/node_modules/html-to-text": { // scope
    "lodash-es": "/node_modules/html-to-text/node_modules/lodash-es/lodash.js",
    "lodash-es/*": "/node_modules/html-to-text/node_modules/lodash-es/*",
    "lodash": "lodash-es",
    "lodash/*": "lodash-es/*"
  }
}

• When looking for an alias, matching scopes are processed from longest to shortest
  • This means the root scope "/" is always processed at last
• Order in objects doesn't matter
• alias are matched by specificy. lodash/* < lodash/inner/* < lodash/inner/request
• alias must match exactly except when /* is appended.
• When successfully matched the URL is replaced and the aliasing starts again
  • You can use a chain of aliases like with "lodash": "lodash-es"
• Technically this proposal doesn't know about "modules". It's just a aliasing for URLs. Which is more browser-like in my opinion.
• It's possible to use extension-less requests like ../app/file in your application, because aliasing them is supported: "/my/app/file": "/my/app/file.js"
• It's easy to merge multiple alias maps, i. e. npm generated map with your application custom map.
• It's easy to understand and implement. See following snippet.

A very simple implementing could look like this:

// Scope: { key, rules }
// Rule: { key, alias }

const scopes = ...;
// scopes ordered by key.length
// rules ordered by specificy

function process(origin, request) {
  for(const scope of scopes) {
    if(origin.startsWith(scope.key)) {
      for(const rule of scope.rules) {
        if(rule.key.endsWith("/*")) {
          if(request.startsWith(rule.key.substring(0, rule.key.length - 2))) {
            return process(origin, rule.alias + request.substring(rule.key.length - 2));
          }
        } else {
          if(request === rule.key) {
            return process(origin, rule.alias);
          }
        }
      }
    }
  }
  return request;
}

A real implementation could be more efficient. A real implementation should also prevent circular aliasing and throw instead.

Possible concerns compared to the original proposal:

• It's more noisy
  • True, I'm assuming this format is written by tooling and compressed via gzip
• It's no longer human-editable
  • True, when using node_modules the number of modules is huge anyway, so you really want to use a tool for this.
  • A human-editable format could be transpiled to this format.
  • It's human-editable for small applications which don't use node_modules.
  • A tool-generated file could be merged with a application-specific custom file

Possible additions:

Load scopes on demand

{
  "/node_modules/html-to-text": "/node_modules/html-to-text/alias.json"
}

Some parts of the alias mapping could be loaded when needed by passing an URL to the alias file as scope. This would reduce the size.

Relative URLs

Currently only absolute URLs are used. Relative URLs could resolve to the browser rules relative to the JSON file. This would make this alias file location-independent.

What do you think about this?

E.g., show explicitly (using import statements, not just tables) that this works:

import _ from "lodash/core.js";

This looks so cool! Are there any implementations of "package-lock.json -> package-name-map" translators yet? (Is that what I should be thinking of?)

(Edit by @domenic: at the time this comment was written the proposal was known as "package name maps". I've renamed the title to be "import maps", but left the body of subsequent comments alone. Sorry if that's confusing!)

This just came up from the discussion around whatwg/html#3871.

If I have the following:

<script type="packagemap" src="package-map.json"></script>
<script type="module">
  console.log('Exec first module script');
</script>

<script type="module">
  // (where x.js has no dependencies itself)
  import "./x.js";
  console.log('Exec second module script');
</script>

<script type="module">
  import "x";
  console.log('Exec third module script');
</script>

Which module script executions will be blocked by the package name map load?

All of them? Or just the last one?

What is allowed in path_prefix and a package's path? Can it be another domain to load JS hosted elsewhere? Can a scope specify a root path or another domain so as to bypass the implicit URL composition? This seems important to make it clear two scopes can reference the same package (e.g. scope A and scope B can both refer to Cv2 via the same URL). I read through the spec and I initially thought that due to the implicit URL composition this would be impossible, but I asked @shicks his thoughts and he had a looser interpretation of what was allowed in those fields. Might be best to be a little more explicit here even in this early draft. Even if it is along the lines of "We're not exactly sure what should be allowed in these fields, but one could imagine using another domain to reference externally hosted code. See #10.".

Context

I've recently built a tool cdn-run designed to help solve this problem in a very specific context: a browser-based code editor (see: next.plnkr.co). This tool will take a set of top-level dependencies and a package metadata resolver function and will generate a SystemJS configuration suitable for executing code in the context of the requested set of dependencies.

Roadblock

In testing this approach, I came across @angular/core/http in Angular that relies on npm module resolution semantics via deep package.json. This means that creating the static SystemJS configuration for this module is impossible without peeking into its contents. Given the similar objects of package name maps, I wonder if this will also be an issue here.

Specifically, consider @angular/core and @angular/core/http (click links to see package contents). While the former can easily be represented in a package name map, how would the latter be represented?

Problem in the context of package name maps

What are the semantics for two package specifiers where one is a prefix of the other? In node, we rely on the module resolution algorithm consulting package.json to dereference the intended entry point at each level of the tree, but it is unclear to me how this would work in the proposed spec.

StealJS is a systemjs fork that does all that successfully maybe you can exchange some code as i like your repo style more https://stealjs.com/docs/index.html

Is there a specific reason for the use of <script> instead of <link>? The only reason I can think of is the HTTP Header support that comes with <link>, but am unsure what is problematic about that.

I understand nested scopes are there to support nested node_modules, but it is possible to supported nested node_modules fine with a single-level scope with rewriting.

For example:

{
  "path_prefix": "/node_modules",
  "packages": {
    "html-to-text": { "main": "index.js" },
  },
  "scopes": {
    "html-to-text": {
      "path_prefix": "node_modules",
      "packages": {
        "lodash": { "path": "lodash-es", "main": "lodash.js" }
      },
      "scopes": {
        "lodash": {
          "path_prefix": "node_modules",
          "packages": {
            "lodash-dep": { "main": "index.js" }
          }
        }
      }
    }
  }
}

can be rewritten:

{
  "path_prefix": "/node_modules",
  "packages": {
    "html-to-text": { "main": "index.js" },
  },
  "scopes": {
    "html-to-text": {
      "path_prefix": "node_modules",
      "packages": {
        "lodash": { "path": "lodash-es", "main": "lodash.js" }
      },
     "html-to-text/node_modules/lodash": {
       "path_prefix": "node_modules",
       "packages": {
          "lodash-dep": { "main": "index.js" }
        }
     }
  }
}

The main feature that seems to be lost is the ability to "copy and paste" sections of the scope map around arbitrarily, but considering these cases are all machine-generated do we really need to support such a complex feature for only this benefit?

There is some brevity provided, but I find myself I can read a flat JSON data structure much more easily than a nested one as well.

This package map implies a particular frontend workflow whereby dependencies are traced by some process to generate the manfiest, which can then used by the browser. A server can then serve resources without needing to know exact resolutions of package boundaries, with the client providing those resolutions locally.

Performance considerations should be seen as a primary concern - so I'd like to discuss how this information asymmetry between the client and server will affect preloading workflows.

We've tackled these problems with SystemJS for some time, but instead of jumping to our solutions I want to aim to clearly explain the problems.

Edit: I've included an outline of how the SystemJS approach could be adapted to module maps.

Firstly, using the Link modulepreload header on servers with plain names like "lodash" will not be suitable when it is up to the client to determine where to resolve lodash.

Thus, any workflow using this approach will need to use client-based preloading techniques.

The cases for these are:

1. Top-level <script type="module"> in an HTML page, varying between different HTML pages of the app.
2. Dynamic import() statements in code.
3. new Worker('x.js', { type: 'module' }) instantiations

All of these cases should support the ability to preload the full module tree to avoid the latency waterfall of module discovery.

The best techniques likely available in these cases will be:

1. <link rel=modulepreload> for top-level module scripts
2. Some kind of dynamic <link rel=modulepreload> injection for dynamic imports, using a custom JS function
3. This same sort of dynamic <link rel=modulepreload> injection for workers.

There are problems with all these techniques:

1. <link rel=modulepreload> information is now inlined into the HTML page itself (not a separate file). This means any resolution changes result in HTML changes, even if the manifest itself is a separate file. Also the preload information may be the same between different pages of an application but must be repeated in the HTML resulting in duplication.
2. A dynamic JS-based module preloading method is fine, but usage might look something like: dynamicPreload('/exact/path/to/lodash.js'); dynamicPreload('/exacct/path/to/lodash-dep.js'); import('thing-that-uses-lodash');. In this scenario we have now tied the exact resolution of the dependency graph to a source in the dependency graph, which is pretty much the same sort of work needed to inline our resolutions into the module specifiers to begin with. We lose a lot of the caching benefits the package name maps provided in the first place of having resource caching independent of resolution caching. We are also duplicating a lot of tree information between dynamic imports in the app, and creating code bloat.
3. In cases where dynamic imports import dynamic specifiers - import(variableName) - it is not possible at all to inline the preloading information into the source, possibly making these scenarios worse for performance.

So my question is - how can these techniques be improved to ensure that the workflows around package name maps can be performant without losing the benefits?

Please also let me know if any of the above arguments are unclear and I will gladly clarify.

This doesn't necessarily contradict anything in the main proposal, because I'm not exactly sure what is meant by:

Inserting a <script type="packagemap"> after initial document parsing has no effect.

But just in case...

Allowing the packagemap script tag to optionally be dynamically generated by JavaScript would open up quite a bit of flexibility. I would think that typically, this would happen in the head tag of the opening index.html. If it is done there, would this qualify as being done before the initial document parsing has been done?

Use cases would be

1. Pointing to different builds -- customizing referenced libraries based on browser capabilities. This would allow more modern browsers to benefit from smaller downloads (and earlier real adoption, as libraries won't need to wait for the lowest common denominator browser to support the feature before removing the down level compiling). Doing this on the server would be quite difficult, as it would need to maintain a complex lookup between versions of the browser, and which JavaScript capabilities it natively supports.
2. Different references between dev and production. This could more easily be done by the server, but complicates caching strategies.
3. Auto generating large lists of packages based on some wild card rules (e.g. paper-*).

I'm just thinking practically about the package fallbacks.

For avoiding connection issues I would likely want my package fallbacks to check say 2 different CDN sources.

Now consider two packages:

{
  "packages": {
    "a": ["https://cdn1/a.js", "https://cdn2/a.js"],
    "b": ["https://cdn2/b.js", "https://cdn2/b.js"],
  }
}

For this scenario the following questions come to mind:

1. Are we then repeating the CDN URLs for each and every package? This seems redundant but could be ok, although perhaps there could be a way to do this through "path_prefix" rather. That may hinder sharing this technique with polyfills though.
2. Say "a" is requested, and "cdn1" is unavailable, so that the browser has now established a connection to cdn2. Will the request for "b" still try to create a connection to "cdn1" again? That is, are we ensuring maximal connection sharing in this process? This should probably be something that is specified into package maps carefully if this scenario is being catered to.

For me modulemap is more relevant name than packagemap because we are talking about ECMAScript modules, using <script type="module" ... > and usually refer to /node_modules.

If someone had a package with multiple entry points (as rxjs does), how would someone go about setting those up?

e.g.

import { Observable, fromEvent } from 'rxjs';
import { map, filter, mergeMap } from 'rxjs/operators';

Where operators is from operators/index.js under rxjs.

I presume I'm just missing something. Thanks!

Preferably including a JS implementation, with at least rudimentary smoke tests.

I've got somewhat of a start on this, and will update this thread as I make progress. Alternately, help would be appreciated if someone else wants to do the work.

Will package map be used for script tags with bare src?

<script type="module" src="moment"></script>

What would the pros and cons be of this living in a <link rel="manifest" href="/manifest.json" /> file?

I'm using this to build browser compatible modules by converting bare import specifiers to UNPKG URLs.

For unpkg, because package name maps or module import maps aren't supported in browsers yet, unpkg will rewrite any bare import specifiers to full urls if the url of the module requested is suffixed with "?module", so my current package name map is as follows

    "path_prefix": "https://unpkg.com/",
    "packages": {
        "@polymer/lit-element": "@0.6.2/lit-element.js?module",
        "ace-builds": "@1.4.1/src-noconflict/ace.js?module",
        "ace-builds/src-noconflict/ext-language_tools.js": "@1.4.1/src-noconflict/ext-language_tools.js?module",
        "ace-builds/src-noconflict/snippets/snippets.js": "@1.4.1/src-noconflict/snippets/snippets.js?module",
        "dedent": "@0.7.0/dist/dedent.js?module"
    }
}

which works, but means I have to include paths like "ace-builds/src-noconflict/ext-language_tools.js" explicitly, so it would be nice if I could use

{
    "path_prefix": "https://unpkg.com/",
    "path_suffix": "?module",
    "packages": {
        "@polymer/lit-element": "@0.6.2/lit-element.js",
        "ace-builds": "@1.4.1/src-noconflict/ace.js",
        "dedent": "@0.7.0/dist/dedent.js"
    }
}

which would also work with any other path within any of those packages without having to be explicitly declared, but this could be a pointless feature since when this is working in browsers, packages imported from unpkg.com won't need ?module as a suffix anymore anyway.

I fail to see the usefulness of this proposal. For anything other than demos with 3 imports, you probably want a tool that generates this mapping. If you need a tool to run on the graph every time the code has changed you might as well just use a tool that rewrites the path in the JS files, without needing for extra config elsewhere: no extra files to fetch, no extra tags in every HTML page.

If your intent is to avoid tools: you can't. You still need a minifier, so just add a path-rewriter tool in there and you're good to go.

The scope concept is looking really great, but this line worries me a little:

Notice also how the full URL path of the nested lodash package was composed: roughly, top "path_prefix" + scope name + scope's "path_prefix" + package's "path". This design minimizes repetition.

Repetition is not a bad thing if it helps clarify understanding. Web developers are going to spend a lot of their lives banging their heads against this manifest when it isn't working, we should aim to remember that.

Specifically, if I wanted a scope to redirect to an entirely different location, say even another server, that doesn't seem possible with the current composition rules given the above.

I'd like to suggest we open up the path_prefix for scopes to be a base-level relative URL, just like the top-level path_prefix. Allowing the repetition brings both predictability and more flexibility.

Doesn't the suggested approach add too much overhead? As a developer I'm concerned about the following:

• Extra script tag (with new value of attribute)
• Extra HTTP request (that can block all subsequent imports)
• Extra efforts to keep everything in sync (package-map and source code)

I'd like to suggest bundling package resolution info into the file itself. Browsers already use the similar technique when resolving sourcemap url from meta comment:

//# sourceMappingURL=/path/to/sourcemap.map

if we introduce the packageMappingURL comment, the import can look like this:

//# packageMappingURL:moment=/node_modules/moment/src/moment.js

import moment from "moment";

In that case

• no need for extra tag
• no extra request
• easier to sync as everything in the same file (e.g. replace moment with date-fns)

What do you think?

Ryan dahl regrets that require("name") was not specific enough.

Thoughts ?

https://www.youtube.com/watch?v=M3BM9TB-8yA

As mentioned in "A convention-based flat mapping", it'd be ideal to also support a way of doing a very simple package map for applications which are laid out ahead of time in a conventional way on-server. For example, you could imagine something like

{
  "imports": {
    "*": "/node_modules_flattened/$1/index.js"
  }
}

In offline discussions, @nyaxt cautioned that this would add a lot of complexity to what was so far a simple format. For example, as-written this would presumably work at any level of the tree. Still, it'd sure be nice...

An alternative is to not reuse the same format, but instead come up with something specifically tailored for this use case, that only works at top level. As a straw-person, you could do:

{
  "pattern": "/node_modules_flattened/*/index.js"
}

This is an issue is to discuss the potential to have multiple package name maps within a single document. As stated in #1 (comment) it is important to ensure that the application developer has full control over all package name maps.

In particular, there seems to be some contention about if the package name map must be a single resource/file. I would like to discuss having a single package name map per resource, but not a single package name map per document.

There are a few use cases that investigating would be good when talking about this:

Progressive loading of maps

As a document loads it may have a desire to inline a package name map, just like how documents inline <script> content for the initial render as well. If replacement of standard module as shown in the readme is desirable, this may be to simply replace the standard modules with replacements such as polyfills separate from the application modules not needed for first render.

Compose "scope" of packages in map rather than define entirety

The "scope" mechanism is setup as a means to allow control of imports in a nested manner based upon pathname boundaries. This requires management always be done as a whole rather than on a per package basis. This is problematic when using 3rd party scripts. Allowing the ability to reference a separate module map would alleviate this management burden. However, care needs to be taken that subresource integrity is preserved across all loads in order to prevent the 3rd party from being able to change the separate module map contents.

Cache subsection scopes

In a large module map we can imagine having several hundred different entries in a package name map. Being able to replace smaller parts of the package name map rather than the whole seems appealing. Doing so requires having separated caching for subsections, similar to how ESM has separate caching per module.

Contained package name map expectations

Allowing resources to control their package name map allows for them to declare their expected resolution. It would prevent any dependency hell that could occur from the global package name map from going out of sync with the scope's package name map which could be managed and generated separately.

As noted in "Installing a package name map", we're not set on the idea of <script type="packagemap">, and we don't know yet exactly what to do for workers (#2).

The advantage of <script> is that it allows you to embed arbitrary data (like JSON) inline in your document. Since this is critical-path information that will be necessary before you can really get started on your module graph, that's definitely something we want to encourage. Or, is HTTP/2 push good enough for this case?

Another alternative is the <link> + Link: header. This doesn't allow embedding, but the browser can start downloading before it even sees the response body (when using the header). And it works fairly naturally for workers as well (see #2).

Any other interesting alternatives?

Would it make sense to include the JSON config in the manifest.json file?

One cool feature of package-name-maps could be an alias to the current module.

Let's say you would have the following folder structure:

└── src
    ├── components
    │   └── atoms
    │       └── date-picker
    └── util
        └── date

To import date from button you have to write a quite unreadable import:

import date from '../../../util/date':

However with package-name-maps this could be improved a lot!

package.json

{
   "name": "fancy-app"
}

package-name-map:

{
  "path_prefix": "/",
  "packages": {
    "fancy-app": { "path": "." }
  }
}

Now the same import in date-picker is easier to read and still works if copied to another file:

   import 'fancy-app/src/util/date';

There could be a use case for multiple package name maps, where the package name map is effectively overridden by successive definitions (if there are conflicts).

The README currently discusses this feature from a perspective of isolation, but it would be interesting to consider composabililty use cases instead here.

As noted in one of the examples, we could choose a default value for "main", such as ${packageName}.js or index.mjs or similar.

Doing so seems fraught, in that the browser would be picking preferred file names and extensions for your files. Apart from favicon.ico, and maybe a few other cases I'm forgetting, the browser has never done this before---for good reason, I think.

(What about index.html, you might say? Nope, the browser knows nothing about that convention. That's purely the server choosing to respond to x/ with the on-disk-file x/index.html.)

But, I wanted to have a dedicated issue to discuss this.

Given:

{
  "path_prefix": "/node_modules",
  "packages": {
    "moment": { "main": "src/moment.js" }
  }
}

import 'moment?tz=utc';

It seems like it should expand to /node_modules/src/moment.js?tz=utc. It seems there might be some missing behavior allowing this given some comments in nodejs/node#20134 (comment)

I like this proposal! I also like that it's fairly trivial to get npm itself to programmatically generate this sort of file automatically, which we'd be happy to do if this lands <3

I have a question which can also be a suggestion: Is there a reason I'm missing for scope existing as a separate key, instead of having a package key nested within individual package entries?

That is, instead of:

{
  "path_prefix": "/node_modules",
  "packages": {
    "redux": { "main": "lib/index.js" },
    "html-to-text": { "main": "index.js" },
    "lodash": { "path": "lodash-es", "main": "lodash.js" }
  },
  "scopes": {
    "html-to-text": {
      "path_prefix": "node_modules",
      "packages": {
        "lodash": { "path": "lodash-es", "main": "lodash.js" }
      }
    }
  }
}

Representing that as:

{
  "path_prefix": "/node_modules",
  "packages": {
    "redux": { "main": "lib/index.js" },
    "html-to-text": {
      "main": "index.js"
      "path_prefix": "node_modules",
      "packages": {
        "lodash": { "path": "lodash-es", "main": "lodash.js" }
      }
    },
    "lodash": { "path": "lodash-es", "main": "lodash.js" }
  },
  "scopes": {
    "html-to-text": {
      "path_prefix": "node_modules",
      "packages": {
        "lodash": { "path": "lodash-es", "main": "lodash.js" }
      }
    }
  }
}

This proposal covers resolving module specifiers from JS imports and exports, but it doesn't cover an adjacent issue of how to utilize package names from HTML or CSS.

This is important when you're loading resources from a package like stylesheets or non-module script:.

In HTML, "bare specifiers" aren't reserved like they are in JS imports, and are interpreted as relative paths. We need some separate scheme, maybe package: to indicate that a path should be resolved though the map:

<link rel="stylesheet" href="package:prismjs/themes/prims.css">
<script src="package:prismjs/prism.js"></script>

In CSS we have the url() function, which could either support the new scheme, or we could add a new function, maybe like import(), to support names:

.foo {
  background: import('bar/bar.css');
}

This makes sense for an app's direct dependencies.

But what about dependencies the dependencies have, and I have no idea that they are there?

Imagine a npm listing.

• Depth = 1 are my direct dependencies. OK. My responsibility to map them.
• Depth > 1 are lurking dependencies I don't know even exist! Who's responsible for their mapping?

I could easily end up with the same package, same version but from different paths!

This is a serious, serious issue ESMs have to deal with. Glad to see it discussed here.

It could be useful to define "environment variables" which conditionally define resolution paths in package maps.

Something like:

<script>
  let legacy = false;
  if (!featureDetection())
    legacy = true;
  packagemapEnvironment.set('legacy', legacy);
</script>
<script type="packagemap">
{
  "packages": {
    "lodash": {
      "legacy": "/lodash-legacy-browsers.js",
      "default": "/lodash.js"
    }
  }
}
</script>

Consider the following case:

• We have a package foo
• foo depends on bar, baz, and qux@1
• Both bar and baz depend on qux@2

In this situation, the tree that npm and Yarn will currently generate looks like this:

/node_modules/Foo/node_modules/Qux
/node_modules/Bar/node_modules/Qux
/node_modules/Qux

They cannot hoist the qux you see in foo and bar, because it would then conflict with the version requested by foo. As such, it needs to be duplicated on the disk, even though it's the exact same files. Because of this, Node will also duplicate them in memory (there will be two instances of qux@1), and I think the same thing will happen here according to this specification.

Given that this situation actually happens in real life and that the hoisting is a purely optional mechanism (a package manager could choose to completely disable the hoisting, for example), I think it would be nice to design a way to say that a file should be instantiated a single time for each time it can be found in the tree (there's a catch: peer dependencies, but I'll come to that later).

(Related to #6)

If a package name map is being requested, then fetching of bare modules waits for the package name map fetch.

I feel this would require resolve a module specifier to be async, so that it can wait until package name load completion.

My random thoughts:

(I feel Option 1 and Option A look good but haven't investigated the issues around them thoroughly; also there can be other options)

Called from fetch the descendants of a module script

Option 1

• Make resolve a module specifier async.
• Make internal module script graph fetching procedure to
  • take a specifier (instead of url) as an argument,
  • call resolve a module specifier asynchronously (instead from fetch the descendants of a module script Step 5),
  • then check visited set (instead in fetch the descendants of a module script Step 5),
  • then proceed to Step 2.

pros:

• As internal module script graph fetching procedure spec is written in a highly async fashion, I expect the changes to spec and implementation can be done relatively easy, without breaking user-visible behavior.
• This will allow fetch a module script graph to take specifier as well, i.e. allow <script> to take specifier.

cons:

• This still requires structural change (one additional async step in internal module script graph fetching procedure, move visited set across spec concepts), and thus changes to spec/implementation will be non-trivial.
• Especially, we have to be very careful about moving visited set check (e.g. to avoid causing performance/non-determinism issues again).

Option 2

• Make resolve a module specifier async.
• Just update fetch the descendants of a module script around Step 5 to allow resolve a module specifier to be async, i.e. make Step 5 to wait for all resolve a module specifier calls and then proceed to Step 6.

pros:

• Smaller changes.
• quite simple, easier to reason.

cons:

• Slows down non-bare specifiers when a module script contains both non-bare and bare specifiers.

We might want to start internal module script graph fetching procedure for child module scripts with non-bare specifiers, and wait for all other bare-specifiers resolution and start internal module script graph fetching procedure for child module scripts with bare specifiers.
However, in this case I expect we still have to reason about visited set check changes, and have to consider similar things to Option 1.

Option 3

• Keep resolve a module specifier sync.
• Make resolve a module specifier return URLs if successfully resolved without waiting for map loading,
  and return bare specifier as-is if package name file is still loading, and let the subsequent internal module script graph fetching procedure to wait/block for actual resolution.

pros:

• resolve a module specifier is sync.

cons:

• Reasoning is harder than Option 1/2, due to mixture of URLs and bare-specifiers returned by resolve a module specifier. If we make resolve a module specifier always return bare-specifiers, then it would be virttually the same as Option 1.

Called from create a module script

Option A

Still require create a module script to throw an error if the specifier is wrong (for bare specifiers, there's no package map, or resolution with the package map fails).

pros:

The semantics for users will be clear.

cons:

Have to make create a module script async (unless Option 3 is selected), which looks a little awkward. (Not so hard, as create a module script is basically in the middle of async call chains in the spec; not so sure about implementation changes though)

Option B

Just do some sanity checks in create a module script (i.e. do not check bare specifiers if the package map is still loading).

pros:

Implementation might be easier?

cons:

Whether create a module script throws or not becomes non-deterministic.
Exposure of this non-determinism to users can be prevented by tweaking find the first parse error, but this non-determinism makes reasoning harder and I'm not sure whether this non-determinism is exposed to users in corner cases.

It's unclear how to apply import maps to a worker. There are essentially three categories of approach I can think of:

• The worker-creator specifies the import map, e.g. with new Worker(url, { type: "module", importMap: ... })
• The worker itself specifies the import map, e.g. with self.setImportMap(...) or import "map.json" assert { type: "importmap" }.
• The HTTP headers on the worker script specify the import map, e.g. Import-Map: file.json or maybe even Import-Map: { ... a bunch of JSON inlined into the header ... }.

The worker-creator specified import map is a bit strange:

• It's unlike the Window-context mechanism in that the creator of the realm controls the realm's module resolution, instead of the realm itself
• It's like the Window-context mechanism in that the script that runs in the realm does not control the realm's module resolution; the control is external to the script itself.

Basically, there is no clear translation of <script type="importmap"> into a worker setting.

Also, as pointed out below, anything where the creator controls the import map works poorly for service worker updates.

The worker itself specifying seems basically unworkable, for reasons discussed below.

And the header-based mechanism is hard to develop against and deploy.

new Worker(someURL, { type: "module", packageMap: ... });

new Worker(url, {
  type: "module",
  packageMap: JSON.parse(document.querySelector('script[type="packagemap"]').textContent)
});

Introduction

The Chrome team is keenly interested in being able to use package name maps both as a way of bringing the bare-import-specifier experience to the web, and as a way of enabling web platform features to be shipped as modules (the layered APIs project). In particular we want to enable the LAPI-related user stories in drufball/layered-apis#34.

The current proposal was created specifically to solve the bare import specifier problem, and is pretty good at that, ongoing tweaks aside. But it only has some tentative gestures in the direction of web platform-supplied modules. The proposed syntaxes are half-baked and feel tacked on to the existing proposal, instead of integrating well with it.

My best attempt to use the current package name maps proposal to solve the LAPI use cases is drufball/layered-apis#33. Its biggest drawback is the introduction of the secondary layeredapi: scheme in addition to the std/x (or @std/x) syntax for importing LAPIs. But we are forced into this awkward situation by the current proposal's separation of mapping import specifiers (the left-hand side) to URLs (the right-hand side).

The below is an alternative proposal that was developed from the ground-up to support both use cases in a unified way. It incoporates ideas from several other open issues and PRs along the way. Note that this is written as an evolution of the current proposal, for discussion and to gather thoughts. I'll probably also write a pull request that replaces the existing README with one implementing this proposal, i.e. as if we'd thought of this proposal in the first place. That'll be easier to read top-to-bottom. But I want to have this discussion first with existing collaborators, for which the below framing is probably better.

Proposal details

URL-based mapping, and the import: scheme

As alluded to in #23, it'd be ideal to have a URL scheme that says "use the package name map to resolve the contents". Let's call that scheme import:.

In the current proposal, the bare import specifiers are thought of as "primary", and import: as an add-on feature. That is, we have two namespaces: import specifiers, and URLs, and the purpose of the package name map is to map between them.

This proposal flips things around. Modules are always imported via URL. A URL is the module's primary identifier. There is just one piece of sugar: in JS import statements and import() expressions, the import: part will get auto-prepended for you, when you use a bare import specifier.

With this in hand, we reframe package name maps to be about URL-to-URL mapping. They are no longer about mapping from the import specifier namespace into the URL namespace. They operate entirely within the URL namespace. And most users will be using them to control the import: URL namespace. But you can also use them to control other parts of the URL namespace, which is useful for LAPI user story (C).

Recursive mapping

Now that we have URL-to-URL mapping, one naturally has to wonder: what happens when you map an import: URL to another import: URL? It recurses, of course!

The key question though is error-handling behavior. If you map an import: URL to some other import: URL which is known not to exist, what happens? In this proposal, the mapping gets dropped, perhaps with a warning in your dev console. This works out really well for the LAPI fallback user story (B), as we'll see.

The left- and right-hand sides of the mapping

We've talked about the above as a URL-to-URL mapping. But it's a bit more complex than that, I think.

The current proposal's setup is about mapping a class of module specifiers to a class of URLs, to support submodules. That is, "lodash": { "path": "/node_modules/lodash-es", "main": "lodash.js" } is designed to map both "lodash" -> "/node_modules/lodash-es/lodash.js" and "lodash/*" -> "node_modules/lodash-es/*".

Even if we change the left hand side to a URL (e.g. "import:lodash") instead of a module specifier (e.g. "lodash"), we want to keep this property.

Furthermore, we want to enable the fallback cases discussed in drufball/layered-apis#34 user story (B), or drufball/layered-apis#5. And personally, I want to do so in a way that isn't tied to LAPIs, and works for all modules; that seems way better if we can.

The solution is to extend the right-hand-side of the mapping to allow multiple forms:

• { path, main } tuples, as today
• strings, which behave as in #52 (i.e. they expand to { path, main } tuples derived from splitting on last path segment of the string)
• arrays of the above, which result in trying each URL in sequence and falling back on network error or non-ok fetching status.

Similarly, the left-hand side keeps its meaning today: it's not only a URL, but also a URL prefix for any submodules.

Examples

Bare import specifiers

Consider the existing examples from this repository. In this new URL-based world, they would be

{
  "mappings": {
    "import:moment": { "path": "/node_modules/moment/src", "main": "moment.js" },
    "import:lodash": { "path": "/node_modules/lodash-es", "main": "lodash.js" }
  }
}

Using the #52 behavior, we can just write this as

{
  "mappings": {
    "import:moment": "/node_modules/moment/src/moment.js",
    "import:lodash": "/node_modules/lodash-es/lodash.js
  }
}

We'll prefer this abbreviated form from now on.

Bare import specifiers with fallbacks

Let's say we wanted to use moment from a CDN, but if that CDN was down, fall back to our local copy. Then we could do this:

{
  "mappings": {
    "import:moment": [
      "https://unpkg.com/[email protected]/src/moment.js",
      "/node_modules/moment/src/moment.js"
    ]
  }
}

LAPI fallbacks, user story (B)

Refresh yourself on drufball/layered-apis#34, then consider this map:

{
  "mappings": {
    "import:@std/async-local-storage": [
      "import:@std/async-local-storage/index",
      "/node_modules/als-polyfill/index.mjs"
    ]
  }
}

This assumes that LAPIs modules are registered (by the browser) at import:@std/lapi-name/*, with an index module in particular existing for each LAPI.

In browser class (1): import "@std/async-local-storage" maps to the URL import:@std/async-local-storage/index which the browser has pre-registered a module for. It works!

In browser class (2): import "@std/async-local-storage" maps to the URL "/node_modules/als-polyfill/index.mjs", after trying import:@std/async-local-storage and getting a failure. It works!

LAPI fallbacks, user story (C)

Refresh yourself on drufball/layered-apis#34, then consider this map:

{
  "mappings": {
    "/node_modules/als-polyfill": "import:@std/async-local-storage/index"
  }
}

In browser class (1): import "/node_modules/als-polyfill/index" maps to the URL import:@std/async-local-storage/index", which the browser has pre-registered a module for. It works!

In browser class (2): this mapping gets dropped from the package name map, per the "recursive mapping" rules above. So such browsers just use the original import statements, pulling in the polyfill. It works!

In browser class (3): the browser doesn't know about package name maps at all, so again the original import statements work, as desired.

Discussion

Overall this proposal accomplishes my goals. It allows package name maps to solve the LAPI use cases, while being more unified; they didn't grow any special capabilities or keys specific to LAPIs. It also solves #23, not in a tacked-on way, but in a way that gets integrated deeply into the mapping.

I see two drawbacks with this proposal:

• Polyfill packages cannot easily have file extensions, especially for submodules. The way in which we do the class-of-URLs to class-of-URLs mapping means that if we want, e.g. import:@std/virtual-scroller/virtual-content to map to /node_modules/vs-polyfill/virtual-content.mjs, we'd need a second mapping, at least.
  • Potential solution: lean into it, and get rid of the class-of-URLs-to-class of URLs mapping entirely? I.e. make everything 1:1, so you'd need to enumerate the submodules of each package, both for LAPIs and non-LAPIs.
  • Potential solution: introduce wildcard substitution? @nyaxt, our Chrome implementer, really doesn't like this path. And it muddles the meaning of these things to be more "URL templates" than "URLs", hindering our ability to reuse infrastructure like the platform's URL parser. So, meh.
  • Potential solution: pick a file extension for the web, and use that for LAPIs? Seems like a non-starter.
  • Potential solution: add an auto-extension-appending feature to the mapping? E.g. { path, main, extension }?
• The amount of ceremony, and concepts to understand, to accomplish LAPI fallback user stories (B) and (C) is high compared to a bespoke LAPI-specific solution. For example having to understand the existence of an index module for each LAPI, or having to use the form "import:@std/x": ["import:@std/x/index", fallback] to express "import:@std/x should fall back to fallback".
  • Potential solution: a dedicated fallbacks top-level section, instead of using array right-hand-sides to the mappings? Still not LAPI-specific, but it is simpler to use.

As an example, if we used the dedicated fallbacks key and the new extension key, a package name map for user story (B) might look more like this:

{
  "mappings": {
    "import:moment": {
      "path": "https://unpkg.com/[email protected]/src",
      "main": "moment",
      "extension": ".js"
    }
  },
  "fallbacks": {
    "import:moment": [{
      "path": "/node_modules/moment",
      "main": "moment",
      "extension": ".js"
    }],
    "import:@std/virtual-scroller": [{
      "path": "/node_modules/virtual-scroller-polyfill",
      "main": "index",
      "extension": ".mjs"
    }]
  }
}

Thoughts welcome, either on these points or more generally.

I'm hoping this scenario has already been thought through, but I just can't find where it's been explicitly stated:

It's wonderful when a library can separate cleanly into individual functions, and applications can pick and choose just what they need. This works best for silo'd applications, of which there's certainly a huge demand. However, particularly in intranet settings, this principle yields diminishing returns, as the

• Size of the organization increases, and
• Despite the large size, there's a desire to keep the library choices constrained, to allow employees to move easily from one group to another
• Teams build their own applications, according to their own release schedule (thus having one giant build is impractical) but
• Their applications are linked together via iframes or regular links.

In this situation, a common CDN is quite useful, with perhaps an incentive to only support major releases of these libraries, so applications will more likely tend to converge on the same version (and help provide some semblance of "governance" over library choices).

The problem is those individual files with individual functions now become problematic. In this scenario, we really need to think of the "lodash" library, or the "date-fns" library, with perhaps a few common packages built from them.

It would be good if the package-name-map could make this easy to manage. Something like:

"lodash": { "path": "lodash/*", "main": "lodash_bundled.js" }

Are there any ideas around building in natively extensibility the way that Webpack builds into imports? Meaning: could there be a way in which a named map (or a matched regex) returns not a path but can point to a function that returns a resource? It would break the JSON format but IMO using JSON is limiting. It should probably be something more imperative like the way Custom Elements are registered, and could then support scenarios like returning JS imports from, say, import styles from "styles.css" or to provide other extensibility.

When finding the applicable scope, what string should we use to do longest prefix matching for the top-level scope?

For example, consider a case given a HTML hosted at http://example.com/baz/app.html:

<script type="packagemap">
{
  "path_prefix": "/foo",
  "packages": {
     "moment": { "main": "src/moment.js" },
  }
}
</script>
<script type=module src="http://example.com/main.js"></script> // should import 'moment' resolve? to which URL?
<script type=module src="http://example.com/foo/main.js"></script> // or would it require this?
<script type=module src="http://example.com/baz/foo/main.js"></script> // or this?
<script type=module src="https://example.com/baz/foo/main.js"></script> // what if html is http page and it references https?
```.