GithubHelp home page GithubHelp logo

letorbi / tarp.require Goto Github PK

View Code? Open in Web Editor NEW
115.0 12.0 27.0 329 KB

A lightweight & asynchronous JavaScript loader for CommonJS and NodeJS modules.

License: GNU Lesser General Public License v3.0

JavaScript 61.85% HTML 38.15%
javascript commonjs module-loader

tarp.require's Introduction

Important notice: Tarp.require is the replacement of Smoothie. It introduces a number of new features and improvements, so it is recommended to use Tarp.require from now on. Please read the migration documentation for further information.

//\ Tarp.require - a lightweight JavaScript module loader

Tarp.require is a CommonJS and Node.js compatible module loader licensed as open source under the LGPL v3. It aims to be as lightweight as possible while not missing any features.

Tarp.require has finally reached a stable state. This means that the version 1.x branch will only receive bugfixes from now on. An improved version of Tarp.require with new features and breaking changes can be found in the tarp2-branch.

Features

  • Compatible NodeJS 9.2.0 and CommonJS modules 1.1
  • Plain No dependencies, no need to compile/bundle/etc the modules
  • Asynchronous Non-blocking loading of module files
  • Modern Makes use of promises and other features (support for older browsers via polyfills)
  • Lightweight Just about 180 lines of code (incl. comments), minified version is about 2kB

Browser compatibility

Installation

The easiest way to install Tarp.require is via NPM:

$ npm install --save @tarp/require

If you don't want to use NPM you can just clone the repository directly or add it to your git repository as a submodule:

$ git submodule add https://github.com/letorbi/tarp.require.git

Usage

Assuming you've installed Tarp.require in the folder //example.com/node_modules/@tarp/require and your HTML-document is located at //example.com/page/index.html, you only have to add the following lines to your HTML to load the script located at //example.com/page/scripts/main.js as your main-module:

<script src="/node_modules/@tarp/require/require.min.js"></script>
<script>Tarp.require({main: "./scripts/main"});</script>

Inside your main-module (and any sub-module, of course) you can use require() as you know it from CommonJS/NodeJS. Assuming you're in the main-module, module-IDs will be resolved to the following paths:

  • require("someModule") loads //example.com/page/node_modules/someModule.js
  • require("./someModule") loads //example.com/page/scripts/someModule.js
  • require("/someModule") loads //example.com/someModule.js

Note that global modules are loaded from //example.com/page/node_modules and not from //example.com/node_modules. This is because the default global module path is set to ['./node_modules'] and is derived from the location of the page that initializes Tarp.require.

Synchronous and asynchronous loading

Tarp.require supports synchronous and asynchronous loading of modules. If you load Tarp.require like described above and use only simple require calls you won't have to care about anything: Just write require("someModule") and Tarp.require will try to load the module and all its sub-modules asynchronously.

However, there might be occasions where you have to force Tarp.require to load a module synchronously. This is possible, but you should try to avoid this, because synchronous requests might block the loading process of your page and are therefore marked as obsolete by now.

Keep also in mind that synchronous loading of a module prevents the pre-loading of its sub-modules. You have to explicitly load a sub-module asynchronously to re-start the pre-loading.

What modules can be pre-loaded?

Right now only plain require-calls are pre-loaded. This means that the ID of the module has to be one simple string. Also require-calls with more than one parameter are ignored.

Example: If a module has been loaded asynchronously and contains the require calls require("Submodule1"), require("Submodule2", true) and require("Submodule" + "3") somewhere in its code, only Submodule1 will be pre-loaded, since the require-call for Submodule2 has more than one parameter and the module-ID in the require-call for Submodule3 is not one simple string.

Enforce synchronous or asynchronous loading

Synchronous or asynchronous loading of a module can be enforced by adding a boolean value as a second parameter to the require-call:

// explicit synchronous loading
var someModule = require("someModule", false);
someModule.someFunc();

 // explicit asynchronous loading
require("anotherModule", true).then(function(anotherModule) {
    anotherModule.anotherFunc();
});

Path resolving

Tarp.require mainly resolves URLs in the same way as Node.js does resolve paths. The only difference is that Tarp.require won't look for a file at other locations if it cannot be found at the resolved URL. This decision has been made due to the fact that modules are usually loaded from a remote server and sending multiple request for different locations would be very time-consuming. Tarp.require relies on the server to resolve unknown files instead.

HTTP redirects

Tarp.require is able to handle temporary (301) and permanent (303) HTTP redirects. A common case where redirects might be handy is to return the contents of index.js or package.json if an ID without a filename is requested. The following NGINX configuration rule will mimic the behavior of NodeJS:

location ~ ^(/node_modules/.*)\.(?:js|json)$  {
    if ( -f $request_filename ) {
        break;
    }
    if ( -f "${document_root}$1/index.js" ) {
        return 301 "$1/index.js";
    }
    if ( -f "${document_root}$1/package.json" ) {
        return 301 "$1/package.json";
    }
    return 404;
}

This will redirect all requests like /node_modules/someModule to /node_modules/someModule/package.json, if /node_modules/someModule is a directory and if /node_modules/someModule/package.json is a file. If that file doesn't exist, the request will be redirected to /node_modules/path/index.js. If both files don't exist, a "404 Not Found" response will be sent.

Note: HTTP redirects won't work in IE11 due to limited support of XMLHttpRequest advanced features.

NPM packages

// The loading of a NPM package is the only occasion when Tarp.require might redirect a request on its own. Tarp.require load a module-ID specified the main field of a package.json file, if the following checks are true:

  1. The package.json file is loaded via a redirect (like explained in the section above)
  2. The response contains a valid JSON object
  3. The object has a property called main

If that is the case a second request will be triggered to load the modules specified in main and the exports of that module will be returned. Otherwise simply the content of package.json is returned.

The module.paths property

Tarp.require always uses the first item of the global paths array to resolve an URL from a global module-ID. Unlike Node.js it won't iterate over the whole array. Since the global config is always used, any change to module.paths won't affect the loading of modules.

Tarp.require also supports the require.resolve.paths() function that returns an array of paths that have been searched to resolve the given module-ID.

Options

Change the global module path

If your modules are not located at ./node_modules/, you can tell Tarp.require their location via the paths option:

Tarp.require({main: "./scripts/main", paths: ["/path/to/node/modules"]});

Override path resolver

If you need a more sophisticated path resolver, you can override the default function that resolves a module-id to the according URL:

Tarp.require({main: "./scripts/main", resolver: function(id, pwd, resolve) { ... }});

The parameter id is the module-id of the module that shall be loaded, pwd is the path of the module require() is called from and resolve points to the build-in path-resolver function (you might want to call this).

Keep in mind that a custom path-resolver may break NodeJS or CommonJS compatibility, if not implemented properly.

Change the document root path

The document root path is used to resolve relative paths inside the paths array. It points to location.href by default. You have to set the document root path explicitly, if you want to use Tarp.require inside a web-worker that has been loaded from a blob.

Tarp.require({main: "./scripts/main", root: "/path/to/page/"});

Using require() directly in the HTML-document

It is not recommended to load other modules than the main-module directly from your HTML-document. However, if you really want to use require() directly in the HTML-document, you can add Tarp.require to the global scope:

Tarp.require({expose: true});

Keep in mind that a require-call in the HTML-document will load a module synchronously unless you explicitly load it asynchronously by adding true as a second parameter.

Load the main-module synchronously

If you really need to load the main-module synchronously, you can do it by loading Tarp.require with the following options:

Tarp.require({main: "./scripts/main", sync: true});


Copyright 2013-2020 Torben Haase <https://pixelsvsbytes.com>

tarp.require's People

Contributors

danvk avatar letorbi avatar lwr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tarp.require's Issues

Path config do not work as aspected

A path config like
<script>var TarpConfig = { require: { paths: "./lib/"} };</script>
do not work in the latest version. Now it's necessary to write
<script>var TarpConfig = { require: { paths: [(new URL("./lib/external/", location.href)).href] } };</script>
Otherwise I get the message:
TypeError: . is not a valid URL.

Does not work on local files

require does not work if the scrips are loaded locally (ie. if loaded from a file:///whatever.html page). This appears to be due to the fact that local files don't specify a mime type and that XMLHttpRequest then assumes that the files are XML and attempts to parse them as such.

Adding request.overrideMimeType('text/javascript'); just before request.open("GET", href, asyn); appears to fix the problem.

Supply code under MIT/BSD licence or equivalent

Hi Torben - we think your browser-based "require" system is really marvellous - it's the smallest version with the tighest and most logical code we found after a long search - many are over 1000 lines :)
We would love to use it in our project. Unfortunately our project rules prohibit the use of GPL3-based code since the viral nature of the licence would end up impacting all our dependencies, and this is the most recent version of the licence we could find for your algorithm.
We'd be REALLY grateful if you'd consider issuing a licence for the code that was MIT/BSD or a similar one - there'd be all of the benefits of ensuring it was preserved in the public domain as well as allowing it to be used in a wider variety of projects which integrate code from different sources.
Many thanks for considering this!
Antranig

http://opensource.org/licenses/BSD-3-Clause
http://opensource.org/licenses/MIT

Can't access node_modules/@tarp when using express.static

This probably isn't a bug, I'm just having trouble getting this to work for my particular website structure. Sorry if the answer is obvious, I've just gotten stuck and could use some help.
Structure:
node_modules
static
--js
----main.js
--index.html
index.js

I put the 2 lines in index.html to load in tarp, but I get this error:
http://localhost:3000/node_modules/@tarp/require/require.min.js net::ERR_ABORTED 404 (Not Found)
I beleive this is because of the following line in my index.js file, which serves up the website:
app.use('/', express.static(path.join(__dirname, 'static')))
This seems to append "static" to the beginning of all server paths, which is great to access my html file, but not for loading tarp. Any suggestions for how to fix this? Thanks!

Fix: make require work with web workers

To make tarp.require work from within a web worker, I had to add following changes:

var locationRef = location.protocol === 'blob:' ? location.origin : location.href;

var pwd = (new URL(id[0] == "." ? (parent ? parent.uri : locationRef) : config.paths[0], locationRef)).href;

When a blob: is detected as a protocol, it means we are in a worker thread and need to make sure we get the correct location.
(was too short to create a pull request)

It works also within workers in a standalone electron application.

"SCRIPT1086: Module import or export statement unexpected here require.min.js (2,170)"

The title shows my current error in the console for an app I'm trying to build.

My HTML is as it is in this Gist, except I have my actual Key in the file on disc: https://gist.github.com/DragonOsman/506bd2bae1dfe4c03a4bbe2a6d830324

And my main script file is called scripts.js, which is as it is here: https://gist.github.com/DragonOsman/c6e8fb15343544e662f474c5a526d1c2

The file country-currency-map.js is a Node module. Please help me fix this issue.

Edit: I also have an error saying:

HTTP404: NOT FOUND - The server has not found anything matching the requested URI (Uniform Resource Identifier).
(XHR)GET - http://localhost:8080/node_modules/node_modules/country-currency-map/country-currency-map.js

 SCRIPT5022: http://localhost:8080/node_modules/node_modules/country-currency-map/country-currency-map.js 404

I have the correct path mentioned in the call to require in my script file. And the script file itself that I'm using as my main script is a client-side script.

can't create parser in IE Worker

Hi Torben,
not sure where Smoothie is going, but I thought you might have a solution for the below:
var parser = URL ? new URL(location.href) : document.createElement('A');
in IE in a web worker.

IE 10 has no support for URL and workers cannot create DOM objects.
As a result, using require from a web worker fails. I can't seem to think of a solution...
Eric

Require relative paths doesn't unwind

I've been considering a simple case to demonstrate, but I'll just post what I noticed for now.

I have a project something like

Voxelarium/Voxelarium.js - requires ./src/voxels.js
Voxelarium/src/voxels.js - required dynamically (./voxes/voxel_###.js) until it fails to find the next in line. (the 6 that are known to work do load correctly)
then Voxelarium.js requires ./src/sector.js

at this point the request that goes to the server is 'http://localhost/src/src/sector.js' (it keeps 'src' in there from when voxels.js loaded a requirement from a relative path...

---Voxelarim.js---

require( "./src/voxels.js" )
require( "./src/sector.js" )

-- Voxels.js ---
require( "./voxels/voxel_1.js" );


In voxels.js I previously was doing

        xhrObj.open('GET', `./src/voxels/voxel_${n}.js`, false);
        xhrObj.send(null);
        eval(xhrObj.responseText);

        //require( `./voxels/voxel_${n}.js` )

which I realized I could just replace with require now that I'm using your module... but then loading started failing.

Async preload do not use the correct path

If I require a script async, then all require in this script, that do not use a relative path will do not use the configured path for preload. e.g.:
/index.html contains:
Tarp.require('./index', true);
index.js contains:
require('my_mod');
This will load the /index.js and than try to load the /my_mod.js. But correct would be to load the /node_modules/my_mod.js.
Later if the my_mod will be used, the correct my_mod will be loaded.

Circular import

test.html:

<!doctype html>
<html>
<head>
<script src="js/require.js"></script>
<script>
require('a');
</script>
</head>
</html>

a.js:

require('b')

b.js:

require('a')

This test case causes chrome to go into swap.

According to nodejs docs, this should never happen because each file is imported only once, the second time it is read from cache.

require a module from node_modules

My HTML file requires a module that requires another module that requires a module from the node_modules folder (which in turn requires some more modules from node_modules).

If I understand the docs correctly, currently the only way to include these modules is to add their folder in the initialization of smoothie, and then refer to these folders with an index, i.e., "1:[module_name]". But, then it will not work with Node.js...

Is there a way to require node_modues that works both in HTML and in Node.js?

HTTP Redirects do not work in IE11

HTTP redirects are not resolved properly in IE11, since it doesn't support the XMLHttpRequest.responseURL.

This might cause the same module to be loaded twice, if it is requested via different module-IDs that point to the same module-URL (e.g. ./foo/../bar and ./bar).

conflict in project where antlr4 uses require and ace used requirejs

H,

I'm trying to use https://github.com/antlr/antlr4-javascript which uses the require from this project and also use ace (git://github.com/ajaxorg/ace.git#master) that is loaded by https://github.com/jrburke/requirejs.

yet the two have the same function name; require. And antlr code has the require throughout it to run on nodejs.

when I put them in separate pages no problem; but in the same page:

SmoothieError: 'require' already defined in global scope

because the requirejs is fighting it

Any way to get these to play together?

require a module from parent folder

The following HTML file:

<script src='require.js'></script>
<script>var module1 = require('module1'); </script>

works OK (on http://localhost). But, when I move the HTML file to a subfolder and change it to:

<script src='../require.js'></script>
<script>var module1 = require('../module1'); </script>

I get the following error:

 SmoothieError: unable to load module1 (404 Not Found)

Maybe the 'require' parser does not support going to parent folder?

`import` cannot be called from a module

Firefox 59 throws an error when an ES6 module is loaded via:

import * as foo from 'fooModule';

This seems to be due to the fact that import statements are only allowed on the top-level of a script, but the module-code loaded by Tarp.require is wrapped into a function.

How can I change the module_path?

I want to add a module_path to require.js with a sub dictionary at the user's dictionary (like /home/Alice/js_modules) and keep the original path as well. How can I do that?

Relative require doesn't, in fact, work

Env:

  • Latest version of chrome
  • latest master standalone

Dir structure:
root/

index.html
js/

node_modules

vuejs-0.8.4

src

main.js
config.js

main.js:

require('./config.js')

index.html:

require('js/node_modules/vuejs-0.7.4/src/main')

Error: 404 loading file:///path/to/file/path/to/file/js/node_modules/vue-0.8.4/src/config.js Uncaught NetworkError: A network error occurred. (see the problem?)

I need this working tonight and will start investigating shortly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.