Comments (17)
ESM can be automatically detected and loaded
last two years of discussion seem to say this isn't possible
It does not include ESM<->CJS interop
since we already have interop and 99.9% of all the node code is not esm i see no reason to not make this part of the goal
transpile ESM to CJS
can't we just leave this in userland
from modules.
mode:esm can be explicitly set by a CLI flag
We should probably split this and/or be more descriptive about this as --mode
differs from the idea of --format
as described in a writeup. mode
essentially is loading a DB of format heuristics (in the example using MIME). format
is declaring the format of an individual resource.
- We need to specify what happens when conflicts occur
i. Which parts of the CLI are affected,--require
comes to mind. - We need to specify what scope is affected when the flag is set (process cwd package boundary, resolved file package boundary)
- Shebang interop considerations.
This is only the first step. It does not include ESM<->CJS interop. It focuses on providing ESM support in Node. Any attempt to mix ESM with CJS or vice-versa will throw an error.
I am opposed to this idea. Some form of interoperability must exist, even if it is purely through an exposed require
function somewhere. In particular import()
will function in Script from day one by JS spec, so I am unsure what this means since you must support CJS->ESM.
My design goals include resource be unambiguous in nature and anything we ship should behave that way from the start.
Goal: Make ESM work in CJS
transpile ESM to CJS (ex babel, webpack, etc.)
I am entirely opposed to having a transpiler be in core if that is the intention. I think this is solved already and doesn't need to be done in core. Transpilers have bugs and evolve over time. If we land it in core we have to preserve lots of compatibility even if it has bugs.
Extension A - target [undefined] - Conversion paths for CJS -> ESM
I think we should leave this outside scope, just like above with transpilers.
Extension B - target [undefined] - Loader Hooks
These already are designed, but we can look at what needs to change.
Extension C - target [undefined] - Loader Extensions
I think APM/Mocks are requirements to some people /cc @mcollina . Existing designs have ways to do this behavior https://github.com/bmeck/node-apm-loader-example .
Extension D - target [undefined] - npm asset interop
I think we should leave this outside scope, just like above with transpilers.
from modules.
I think we would benefit from making the Roadmap extremely terse for right now, potentially starting with the current implementation and working towards the goal of removing the flag
from modules.
ESM can be automatically detected and loaded
Following, @ceejbot's proposal. Her recommended approach involved detecting the first instance of require|import
and setting a global mode flag so all subsequent modules are handled the same way.
This could be overridden from a CLI entry point by specifying a --mode
flag. @guybedford also mentioned adding a similar field to package.json for module loading.
It does not include ESM<->CJS interop
With or without a .mjs
extension? Last I checked importing ESM as .js
does not work.
transpile ESM to CJS
Yes, and that could be documented as a recommendation or left out altogether.
@MylesBorins Feel free to 'trim the fat' or rearrange as you see fit.
This is more of a suggestion to deal with planning. Right now, we're blocking on 'everything that addresses everything should all happen at once'. IMO, it would be more realistic to identify a concrete starting point and flesh out the unknowns over time.
from modules.
Milestone 1, by nature, needs to include a definitive specification of how spec-unspecified behaviors will resolve in esm, with future extensions in mind. Specifically, how module specifiers will resolve by default. Also it must be stated what the goals of consuming ESM (at all) are. Why ESM? What should drive a package or application author to choose ESM over CJS? At milestone 1, is it expected that all esm we load would also be directly loadable in a browser or other ESM compatable runtime, completely unmodified? It should also be noted what the impact of actually shipping with only those features listed in M1 available would be. Doesn't shipping without interop in either direction start to fracture the package ecosystem along module system bounds? Should that be avoided? If so, can M1 actually be shipped? or is it just an internal milestone? (If so, can you really say it's scheduled for 10.0?) Also, why the assumption about what package and cli flags will set goals? AFAIK, I've only seen these in proposals that exist to compete with the disambiguate-via-extension one - your milestone seems to be a bit presumptuous, unless I've missed something.
Milestone 2's a bit confusing. Is this just about loading ESM in a CJS context? I was under the assumption that top-level dynamic import would just support this, either exclusively (only loading esm), or by disambiguating load kind via side channel info (package json or extension). I suppose those decisions must be finalized?
Milestone 3's where I've seen (and participated in) a lot of contention. I honestly believe consensus and implemention on these topics is what's required to ship unflagged and not do harm to the ecosystem, and not before. Moreover, as written it precludes module hooks or loaders (builtin ones, even) from being part of this plan (since they're left in "extension" categories). Based on some discussions, depending on how other things pan out, the most viable routes for interop may involve custom loaders. It's pretty obvious that interop is required for a successful es module deployment, but what's done here directly impacts browser compat/portability conerns (which were hopefully decided on back during M1?). Even just allowing path traversal for looking up modules is enough to break untooled portability. For that reason, making interop opt-in via a package-scoped loader that declares interop behaviors seems reasonable.
To boot, this kinda ignores the current implementation. Node can already parse/load esm (though not via the same specific mechanisms specified in M1) - right now I'm pretty sure the goals are syncing knowledge and coming to consensus on is a short list of concerns around module system interop and portability.
from modules.
following up on what @MylesBorins said, I think a flag that is not experimental but covers ESM in a way we like could be already considered a great success and make migration also easier to opt in.
Stats based on npm modules would be easier to read so that in the future such flag could be dropped once ESM is the default (similar to a use strict migration approach).
Here how I think milestone 1 as MVP could work:
Milestone 1 - target [v10.0] - ESM Support in Node
Goal: ESM modules work in Node without plugins, loaders
- mode:esm can be explicitly set by a CLI flag
- mode:esm can be explicitly defined in package.json
- mode:esm provides a way (import.meta.require or another) to import CJS too
- mode:esm throws error if CJS is imported as ESM
The last point is basically a consequence of the fact ESM module won't contain module
or exports
or this
and neither require
, so that basically it will inevitably throw when imported as ESM.
Those points should be hopefully enough to grant ESM in Node, where omitting the flag would result in regular CJS behavior, where requiring untranspiled ESM would fail same way imported CJS would in mode:esm.
from modules.
@weswigham See @ceejbot's proposal. I'm just attempting to establish a starting point based on what has already been said. None of this is concrete, none of this has consensus.
Also it must be stated what the goals of consuming ESM (at all) are. Why ESM? What should drive a package or application author to choose ESM over CJS?
IMO, this is out-of-scope. The community will decide whether ESM is a path they wish to pursue. We're just here to provide that as option.
Yes, M2 addresses loading ESM in a CJS context. AFAIK, no it doesn't just work. If you set context via flag it will prefer one-or-the-other not both. CJS and ESM are fundamentally different, the former loads dynamic/sync, the latter static/async.
I added Specification
as a requirement for each of the milestones.
I agree M3 is 'here be dragons' territory. Maybe the specifics will become more clear once M1 is reached. Maybe, further experimentation in the wider ecosystem can provide more context.
Based on some discussions, depending on how other things pan out, the most viable routes for interop may involve custom loaders.
I agree. The order of extension is neither implied nor assumed. It could come before or after, or even be required by one of the milestones. I'm just posting them here to establish that they likely will fall within the scope of this project.
from modules.
@WebReflection I agree except for one point
- mode:esm provides a way (import.meta.require or another) to import CJS too
I added this as M3 because require()
works synchronously. Will this cause issues when loaded with ESM? Is it necessary to implement import.meta.require()
an async variant of require()
? If so, would this change require the addition of top-level await
?
That's also the reason why M3 comes later. M1/M2 should be possible with the the existing ES spec.
from modules.
@evanplaice while the npm proposal and implementation are a great place to draw inspiration from I think we would be better served by starting from the implementation that currently exists in node. The road should start with where we are and reach where we want to go
re: import.meta.require
it is run in the body which is executed synchronously in ESM as it is... there is no conflict or need to play with anything... it is also something we could ship today (PR was opened but scaled back)
from modules.
@MylesBorins OK, just to clarify would it best to swap M2 <-> M3 and drop any mention of async loading?
from modules.
TBH I think that this current approach of a roadmap is not starting from the right place and likely needs to be reworked / envisioned. Unfortunately I am at the end of my work day so I can't take the time to respond with an alternative, but what I think we need to do before starting to map things in time is define a couple things
First thing we need to do is define our current implementation
- what are the current features of our implementation
- this is where we start
Once we have this we should then start to collect things that need to be done, these things can be in conflict:
- what still needs to be implemented
- for example modes
- what behavior may have to change
- for example transparent interop (this is editorialized)
this should be done independent of timelines at first
We should then define shipping goals which we've begun to do in #4
The combination of physical timelines and shipping goals can help us start to place the proposed changes that we have identified. Conflicting changes can then be debated at the point that they block work.
from modules.
Cool. I'll step back for a bit and wait to see how some of the other discussions pan out.
from modules.
from modules.
Also it must be stated what the goals of consuming ESM (at all) are. Why ESM? What should drive a package or application author to choose ESM over CJS?
IMO, this is out-of-scope. The community will decide whether ESM is a path they wish to pursue. We're just here to provide that as option.
@evanplaice I'm pretty sure that the feature is being designed for someone, or at least with certain developers and workflows in mind. It's best to be explicit about who these groups are and what their priorities are so you can properly evaluate what tradeoffs to make during design, and how those choices affect each of those groups.
from modules.
@bmeck Following on from what @MylesBorins it's probably premature to address a roadmap for now.
Specifically, how does the current implementation work, what will be the agreed-upon approach going forward, and what changes need to take place to make it work with minimal changes to Node.
mode:esm can be explicitly set by a CLI flag
The details of this approach haven't been fleshed out and require further discussion.
The current approach, using mimetypes to specify module format breaks browser interop. On the Node side .js implies CJS as the default. In the browser .js implies ESM. I'd go so far as to say this is the primary conflict that led to the formation of this working group. The mode
flag is one possible approach.
- Good point
--require
will need to be addressed - Mode can be detected (ie first require/import statement) in the main entry point
- modes can be explicitly defined as an efficient path to override detectiton
- CLI works the same
- pkg.mode only affects how pkg.main (ex /index.js) is loaded
- package or cwd relative paths are ambiguous because libs may provide both CJS and ESM compatible imports
- Shebang also needs to be addressed, could it be used with the CLI flag to specify mode
Keep in mind the fundamental differences between CJS and ESM. CJS by default encourages deep linking directly to source files because that's the only way to specify imports.
With ESM, the usage of facades (ie index.js files containing re-exports) to expose a public API make directory structure irrelevant. For example, RXJS is currently (ie 6.0) undergoing a massive restructuring that uses facades because users deep linking to internals has caused a lot of support pain for the maintainers.
This is only the first step. It does not include ESM<->CJS interop. It focuses on providing ESM support in Node. Any attempt to mix ESM with CJS or vice-versa will throw an error.
ESM as it's currently used in the browser is incompatible with how it's used in Node. IMO, addressing this issue is a good starting point; even if the functionality is hidden behind an experimental flag until interop is addressed.
Either way, using require()
to import an ESM should throw an error. Using import
to import CJS should throw an error.
transpile ESM to CJS (ex babel, webpack, etc.)
Of course this shouldn't be added in core. I just brought this up because lib authors may find it preferable to provide both. In the FE, ecosystem isn't common for lib devs to provide alternative formats (ex /dist/[name].umd.js, /dist/[name].esm.js) and tools already exist to accomplish this.
Extension A - target [undefined] - Conversion paths for CJS -> ESM
I agree, the community should be able to provide a solution. I'll mark this as out of scope.
Extension B - target [undefined] - Loader Hooks
Can't say much about this as I haven't used it. What I can say is, if loader hooks are based on directory boundaries, they will not work with emerging ESM based workflows.
Facades will become a common pattern for libs to expose public APIs. The FE ecosystem is already beginning to shift in this direction. In addition, expose * as
syntax will likely land in a future spec providing even greater flexibility for defining facades.
Not to mention the likely introduction of decorators in the future. Decorators will allow functions to be wrapped with additional functionality to address specific concerns and/or define alternative forms of execution for different environments.
I'm not saying loaders aren't useful, just that the current approach to using loaders in CJS isn't compatible with ESM.
Extension D - target [undefined] - npm asset interop
I agree. I'll mark this as out of scope.
from modules.
Could this be closed in lieu of #23?
from modules.
@MylesBorins I'm good with closing it. I'm really liking the direction that 'goals' is going. If it seems beneficial, another draft can be created to map implementation specifics once we have a better idea of how that'll look.
from modules.
Related Issues (20)
- Mocking use cases HOT 13
- Enabling import meta resolve by default? HOT 16
- Relaxing the '#' restriction on imports HOT 8
- Node.js Modules Team Meeting 2020-09-09
- Node.js Modules Team Meeting 2020-09-23 HOT 1
- Document node: scheme HOT 4
- Clarification for bundlers re "import" / "require" matching HOT 7
- Node.js Modules Team Meeting 2020-10-07 HOT 3
- File URL normalization changes HOT 1
- s/master/main HOT 3
- Node.js Modules Team Meeting 2020-10-21 HOT 4
- Experimental Status HOT 18
- Node.js Modules Team Meeting 2020-11-04
- Archiving modules repo and stopping team meetings HOT 1
- Use existing context for SourceTextModule evaluation? HOT 2
- Node.js Modules Team Meeting 2020-11-18
- import local files fails while import() succeeds HOT 5
- Proposal: ESM should not export PromiseLike instance HOT 5
- `import.meta.url` is `undefined` inside module file. HOT 3
- Extensionless files should allow --input-type (or work) HOT 14
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from modules.