Comments (21)
Full URLs seem easiest
Easiest if specifying a generalized preload manifest yes.
but if there's a nice way to ensure (perhaps with tooling?) that the two stay in sync, that would make sense.
So picturing the a user workflow for the best performance, we're running some kind of npm-generate-package-map app.js and.js other-entry-points.js
which is then generating "preload.manifest" and "package.map" for us. I guess that absolute URLs with a leading /
would be ok in such a preload manifest though? Then we're adding those two manifests to the appropriate HTML pages.
That can work definitely, it would just be good to see real momentum on such a spec and a commitment to the production performance use case from this spec, to ensure these problems are solved.
I'm not sure what your concern is about performance from the get-go. Modules are an incremental project; browsers have been shipping pieces of them for a long time, and will continue to ship pieces as they become ready. There has been so far no desire to hold back on shipping module-related features until they are polished to perfection.
This argument doesn't really hold for me. Also never thought I'd hear someone from the Chrome team saying performance doesn't matter :)
Modules have been a development feature, and as such, yes performance has not been a concern.
For production modules features (which it seems like this one is trying to be), performance is a primary concern, and not an afterthought. Let's not fall back into old thinking just because performance hasn't been a concern for modules specs up to now. Production modules features are a very different thing.
If I've misunderstood and the goal is just to have a nice development experience then sure, it's not a concern.
from import-maps.
I must admit this sounds a lot like passing the buck on the performance of this browser feature, which was really not what I was hoping to hear.
I work on these things out of my own free time voluntarily between consulting work, but if it really is going to come down to it then in the name of the health of the web if there is something I can do I can certainly look into it.
I guess something like <link rel="preloadmanifest" href="preload.json">
with a signature of { [url: string]: { rel: string, url: string }[] }
would work. Any suggestions on where best to take this?
from import-maps.
Firstly, using the Link modulepreload header on servers with plain names like "lodash" will not be suitable when it is up to the client to determine where to resolve lodash.
Why is this? A modulepreload header should probably admit package names somehow, allowing the client to then begin fetching the resource as resolved via the package name map.
Even if not, there's also no reason why a server can't utilize the very same map to resolve names to paths and still use a path in the header.
from import-maps.
Thanks for putting some thought to this @justinfagnani.
So the issue is that while yes, the modulepreload can indicate to load Lodash, it can't indicate to load any of the dependencies of lodash in turn - thereby creating a latency waterfall across all package boundaries. Package depth thus becomes a performance anti-pattern.
If the server uses the same map, that means that the same server cannot serve multiple versions of the same application with the same files with far-futures expires on those individual versions.
from import-maps.
It's my understanding that modulepreload already works this way with paths - it loads a module and its dependencies as it discovers them, all driven by the client. If that's correct, how does adding a name resolution step create any more of a waterfall?
I'll have to think about the multi-versioning issue more, but I wonder why the package name map wouldn't be versioned along with the app. In fact, it probably helps with versioning because the map can specify the versioned file paths completely independently of the module source. That way a non-leaf module doesn't need to change if it participates in two app versions with two different versions of dependencies - increasing cacheability.
from import-maps.
@justinfagnani to solve the waterfall problem one needs to inline a modulepreload for each module of the tree depth to avoid the latency problem. Otherwise one is only saving one round trip latency, when really we want to save n round trip latencies, where n is the depth of the tree.
In the package name spec, the n here becomes the package name lookup depth of the graph (eg pkgA depending on pkgB depending on pkgC where those resolutions are defined by the module map).
The aim here being perfect optimal performance - unnecessary latency should always be avoided in the name of keeping the web fast.
Definitely versioned package maps are the way - but the comment was that inlining link tags for deep dependencies means that the link header for a given source would effectively inline a part of the module map - which could then be out of sync with the actual module map - the server could inline one version of lodash's dependencies, while the client expects to use a different version of lodash.
from import-maps.
If there isn't any further feedback on this topic, I'd like to share a draft proposal based on what we've done in SystemJS to speed up the loading waterfall.
from import-maps.
Alright, so here's what we do in SystemJS and how that might apply here. The feature in SystemJS is called depCache
and basically stores the list of unresolved dependencies for each unique module string. Whenever a module is loaded, we then use this information to trigger preloading of the provided dependencies (effectively a dynamic injection of <link rel="modulepreload">
, and this was why I have argued strongly for modulepreload it to be non-recursive to avoid problematic functional complexity here).
The reasoning here is that the build process that constructs the module map, will have all the same information to construct the preloading graph, and thereby avoid a latency waterfall solving the problem of optimized flat loading.
I really think this would be a huge win for module maps to help this performance problem.
Here is a rough idea of how such a thing might work:
Say I have a main script importing pkg1
which the module map resolves, where pkg1
then imports from pkg2
, which again the module map resolves so that preloading information is dependent on the module map.
We could then write something like:
{
"path_prefix": "/node_modules",
"packages": {
"pkg1": { "main": "pkg1.js" }
},
"scopes": {
"pkg1": {
"packages": {
"pkg2": { "path": "pkg2", "main": "pkg2.js" }
}
}
}
"preloadDependencies": {
"pkg1/pkg1.js": ["pkg2", "./pkg1-dep.js"],
"pkg2/pkg2.js": ["./pkg2-dep.js"]
}
}
where the keys of preloadDependencies
are URLs relative to path_prefix
and when matched provide the unresolved specifiers of dependencies to be preloaded. We run those resolutions through the main resolve algorithm again to work out what exactly to preload, then recursively apply preloading again for those modules once resolved.
This way, as soon as import "pkg"
happens or even import("pkg")
, we can ensure that all dependency modules are requested in parallel ensuring minimum latency.
It's a relatively simple feature overall to add I feel for the massive performance benefit to be gained.
from import-maps.
@guybedford Your proposal makes a lot of sense to me. It is the client that ought to manage preloading. The client is already given the receipe/rules to perform resolution in the form of the name-map generated by a tool. So we might as well supplement the name-map with derived data to allow entrypoints to be parallel fetched & parsed rather than involving waterfall discovery.
(Tangent: I'd love to find a way to have eager evaluation of side-effect-free children but alas the spec needs early errors for bad parsing of the full graph before we can start any evaling. Anyway, that's another story.)
from import-maps.
This thread is massive and I haven't had time to read through all of the text in it. But skimming the latest it seems to run afoul of similar concerns to https://github.com/domenic/package-name-maps#supplying-out-of-band-metadata-for-each-module ; that is, this is not meant to be a solution for all modules, and is focused only on package name mappings. A separate proposal, not related to this one, is more appropriate for anything (like preloading) that applies to all modules.
from import-maps.
I'm not suggesting this extend to any other out-of-band metadata here. This could even be restricted to only include package dependencies (pkg2
and ignoring ./pkg1-dep.js
in the example). The package name map is the only source of truth for the deep tree, so it is the only place we can solve the deep preloading through package resolutions. Otherwise latency waterfalls will dominate module workflows.
from import-maps.
I don't think it's fair to say that the package name map is the only source of truth for the deep tree, when I'm specifically pointing out that the intended source of truth for the deep tree is a separate manifest file unrelated to the package name map.
from import-maps.
Perhaps it would help to flesh out how you see these performance problems getting solved then? I originally aimed for an open discussion around these concerns - specifically not to try to jump to solutions here but to discuss the problem space and how we can help users avoid the waterfall, as at the moment it will be the default behaviour. I'd suggest reading the thread though.
from import-maps.
Via a separate manifest file, unrelated to packages, that lists all the modules in the graph.
from import-maps.
I see, do you mean to suggest new spec work for this? Or something in userland?
from import-maps.
Yes, new spec work. Possibly related to https://github.com/WICG/webpackage
from import-maps.
Interesting. Would you expect such a preload manifest to inline full URLs, or might it be able to deal with preloading unresolved module specifiers? Sharing resolution source of truth may cause sync issues between the two. Also I hope whatever route we take modules can be provided performantly from the get-go, also in order not to tarnish reputation when numbers start landing.
from import-maps.
Full URLs seem easiest, but if there's a nice way to ensure (perhaps with tooling?) that the two stay in sync, that would make sense.
I'm not sure what your concern is about performance from the get-go. Modules are an incremental project; browsers have been shipping pieces of them for a long time, and will continue to ship pieces as they become ready. There has been so far no desire to hold back on shipping module-related features until they are polished to perfection.
from import-maps.
The goal is to have a nice developer experience, and to separately continue our efforts to improve performance, so that modules can be used by larger and larger sites in production. Those efforts are orthogonal, so that in the end, you can have a nice developer experience in production.
As for commitment to such a spec, I'd welcome your efforts there; it's not the focus of this repository, or of my work at the moment.
from import-maps.
I do believe the two belong together... Why would these two pieces of information be separate? (and besides apparent logic, I'm also thinking on the "nice developer experience" goal...)
So, if the dependency graph belongs in the said webpackage spec, should the package name map belong there as well?
from import-maps.
As the proposal has changed a lot, I am triaging old issues. I think even with new proposal, the conclusions here still apply. Namely, that this thread is requesting a separate feature, which is not our focus as part of the import maps proposal.
As noted in https://github.com/domenic/import-maps#supplying-out-of-band-metadata-for-each-module, the new proposal is tentatively a bit more connected to the idea of a "module manifest" than the previous one. But, I still lean toward that being a separate proposal, that should not be entangled into this one, for the reasons listed there.
from import-maps.
Related Issues (20)
- Modulepreload error caching HOT 9
- Should HTMLScriptElement.supports("importmap") return true? HOT 1
- Fire a load event or an error event when the script element of the import map is from an external file? HOT 2
- preparation-time document check should be done first at #register-an-import-map HOT 2
- Throwing TypeError in resolve a module specifier? HOT 2
- calling onerror of the script element if the import map string cannot be parsed HOT 1
- What should 'parse an import map string' return when it throws an Error? HOT 8
- Speculatively parsing with import map if there's a dynmic import before, HOT 4
- Official file extension? HOT 2
- Support stars in imports besides trailing slashes HOT 1
- `importmap` cannot resolve dependencies entered relatively. HOT 3
- [feat] `script.integrity` and import map HOT 2
- Specification is not restricting address from containing query- or fragment parts not ending with "/" HOT 3
- global variables (f.e. map `"jquery"` specifier to `globalThis.jquery` variable) HOT 1
- Specify package.json path in <script>, and then generate the import map from package.json. HOT 7
- Unable to create folder alias to use in further alias. HOT 2
- ability to specify a default extension for extensionless URLs HOT 3
- Fallbacks for failed URLs? HOT 1
- import maps for `import()` statements HOT 1
- Support for "package.json browser" style file replacement/removal within modules? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from import-maps.