GithubHelp home page GithubHelp logo

michaelficarra / commonjs-everywhere Goto Github PK

View Code? Open in Web Editor NEW
158.0 10.0 21.0 273 KB

:rainbow: minimal CommonJS browser bundler with aliasing, extensibility, and source maps

License: BSD 3-Clause "New" or "Revised" License

Ruby 0.27% Shell 1.85% CoffeeScript 97.88%

commonjs-everywhere's Introduction

CommonJS Everywhere

CommonJS (node module) browser bundler with source maps from the minified JS bundle to the original source, aliasing for browser overrides, and extensibility for arbitrary compile-to-JS language support.

Install

npm install -g commonjs-everywhere

Usage

CLI

$ bin/cjsify --help

  Usage: cjsify OPT* path/to/entry-file.ext OPT*

  -a, --alias ALIAS:TO      replace requires of file identified by ALIAS with TO
  -h, --handler EXT:MODULE  handle files with extension EXT with module MODULE
  -m, --minify              minify output
  -o, --output FILE         output to FILE instead of stdout
  -r, --root DIR            unqualified requires are relative to DIR; default: cwd
  -s, --source-map FILE     output a source map to FILE
  -v, --verbose             verbose output sent to stderr
  -w, --watch               watch input files/dependencies for changes and rebuild bundle
  -x, --export NAME         export the given entry module as NAME
  --deps                    do not bundle; just list the files that would be bundled
  --help                    display this help message and exit
  --ignore-missing          continue without error when dependency resolution fails
  --inline-source-map       include the source map as a data URI in the generated bundle
  --inline-sources          include source content in generated source maps; default: on
  --node                    include process object; emulate node environment; default: on
  --version                 display the version number and exit

Note: use - as an entry file to accept JavaScript over stdin

Note: to disable an option, prefix it with no-, e.g. --no-node

Example:

Common usage

cjsify src/entry-file.js --export MyLibrary --source-map my-library.js.map >my-library.js

Watch entry file, its dependencies, and even newly added dependencies. Notice that only the files that need to be rebuilt are accessed when one of the watched dependencies are touched. This is a much more efficient approach than simply rebuilding everything.

cjsify -wo my-library.js -x MyLibrary src/entry-file.js

Use a browser-specific version of /lib/node-compatible.js (remember to use root-relative paths for aliasing). An empty alias target is used to delay errors to runtime when requiring the source module (fs in this case).

cjsify -a /lib/node-compatible.js:/lib/browser-compatible.js -a fs: -x MyLibrary lib/entry-file.js

Module Interface

cjsify(entryPoint, root, options) → Spidermonkey AST

Bundles the given file and its dependencies; returns a Spidermonkey AST representation of the bundle. Run the AST through escodegen to generate JS code.

  • entryPoint is a file relative to process.cwd() that will be the initial module marked for inclusion in the bundle as well as the exported module
  • root is the directory to which unqualified requires are relative; defaults to process.cwd()
  • options is an optional object (defaulting to {}) with zero or more of the following properties
    • export: a variable name to add to the global scope; assigned the exported object from the entryPoint module. Any valid Left-Hand-Side Expression may be given instead.
    • aliases: an object whose keys and values are root-rooted paths (/src/file.js), representing values that will replace requires that resolve to the associated keys
    • handlers: an object whose keys are file extensions ('.roy') and whose values are functions from the file contents to either a Spidermonkey-format JS AST like the one esprima produces or a string of JS. Handlers for CoffeeScript and JSON are included by default. If no handler is defined for a file extension, it is assumed to be JavaScript.
    • node: a falsey value causes the bundling phase to omit the process stub that emulates a node environment
    • verbose: log additional operational information to stderr
    • ignoreMissing: continue without error when dependency resolution fails

Examples

CLI example

Say we have the following directory tree:

* todos/
  * components/
    * users/
      - model.coffee
    * todos/
      - index.coffee
  * public/
    * javascripts/

Running the following command will export index.coffee and its dependencies as App.Todos.

cjsify -o public/javascripts/app.js -x App.Todos -r components components/todos/index.coffee

Since the above command specifies components as the root directory for unqualified requires, we are able to require components/users/model.coffee with require 'users/model'. The output file will be public/javascripts/app.js.

Node Module Example

jsAst = (require 'commonjs-everywhere').cjsify 'src/entry-file.coffee', __dirname,
  export: 'MyLibrary'
  aliases:
    '/src/module-that-only-works-in-node.coffee': '/src/module-that-does-the-same-thing-in-the-browser.coffee'
  handlers:
    '.roy': (roySource, filename) ->
      # the Roy compiler outputs JS code right now, so we parse it with esprima
      (require 'esprima').parse (require 'roy').compile roySource, {filename}

{map, code} = (require 'escodegen').generate jsAst,
  sourceMapRoot: __dirname
  sourceMapWithCode: true
  sourceMap: true

Sample Output

commonjs-everywhere's People

Contributors

constellation avatar davidchambers avatar devongovett avatar endangeredmassa avatar fulmicoton avatar ghempton avatar jakobmattsson avatar krisnye avatar michaelficarra avatar vendethiel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

commonjs-everywhere's Issues

Option to build a library with multiple modules and NOT define a module entry point.

Let me describe what would be optimal for my situation, and I think many others.

I have a module based library that I'm writing to be used primarily by web pages.

This will be use by multiple web pages which may each depend on a subset of the total modules.

I don't really want to compile the entire library into a single file per web page.

I would prefer to compile the library by specifying a list of modules (without a single entry point) into a single browser javascript file. That file should register a global variable 'require' which enables me to access any of the modules as needed.

Html pages should be authorable with what is equivalent to my entrypoint module being an inline script like so:

<html>
    <head>
        <script src="mycjsify-library.js"></script>
    </head>
    <body>
        <script>
        var myModule = require("mymoduleAlpha");
        myModule.doSomethingCool();
        </script>
    </body>
</html>

Incremental rebuilding time is too slow and takes O(n) time where n is the size of your project

My project is very small right now. 50kb and depends on Sugar.js, so fully built output size is about 280kb.

It is taking about 4 seconds to rebuild on my Core 2 Duo 2.6 ghz machine.

I added some debug and investigated, and I found that almost all of the time is spent in the "escodegen.generate" function.

Your incremental building code seems plenty fast, but the escodegen is killing performance. The problem seems to be that we have to run the escodegen on ALL of the AST tree even though we only changed just a single line in one source file.

The bigger problem with this is that the performance will degrade further as the project grows. My project will grow from 50k source to probably about 1Meg of source, and the "incremental build" time will increase proportionally.

This makes the current solution unfit for a development environment, where the need to test in the browser requires you to have a very quick turn-around between editing, refreshing and testing. Personally, I require a build that takes no longer than a second.

Do you have any thoughts on this, or possible solutions?

One idea: Could we preserve the code output, and maybe have it contain some delimiters that mark the beginning and ending of each individual module. Then we could just escodegen the modules that change and replace their code within the precompiled bundle.

split up src/index.coffee a bit

  • src/module.coffee
  • src/traverseDependencies.coffee
  • src/bundle.coffee
  • src/resolvePath.coffee
  • src/relativeResolve.coffee
  • src/isCore.coffee

add tests

I need to figure out a good testing strategy. Browserify's tests may be a source of inspiration here, if it has any. I'll look into it.

specify additional dependencies

it would be awesome if I could specify a list of additional dependencies to bundle, so I could use dynamic requires. i.e.

cjsify "lib/index.coffee",  { additionalDependencies: "lib/models/*.coffee" }

and then I could do

require "./models/" + name + ".coffee"

If you think this is something worth adding I'll make a PR.

grunt plugin

A grunt plugin would make this part of a more complete workflow. When one gets written, a link to it in the README would be awesome.

circular dependencies

At the moment, looks like circular dependencies aren't supported. While ugly, it's something you run into from time to time.

The modules spec at http://wiki.commonjs.org/wiki/Modules/1.1.1 states:

If there is a dependency cycle, the foreign module may not have finished executing at the time it is required by one of its transitive dependencies; in this case, the object returned by "require" must contain at least the exports that the foreign module has prepared before the call to require that led to the current module's execution.

I got it working by adding an additional require.cache[file] = module$.exports; before the resolved.call in the PRELUDE, but not sure if it's the best approach (and the tests aren't working on my system.)

Support windows

Hi! Nice module, tried to integrate it on my script but on Windows, it compiles like this :

    require.define('/modules\\jumps.ls', function (module, exports, __dirname, __filename, process) {
        require('/modules\\jumps\\top.ls');
        if (!QS('.player-name')) {
            require('/modules\\jumps\\login.ls');
        }
    });

add watch flag to CLI for continuous builds on dependency change

original topic: split dependency traversal out from cjsify to allow for fast, continuous builds

We might need to split up cjsify into traverseDependencies (or something) and bundle. This will allow us to give it an entry point of a single changed file, get back a mapping from new files to ASTs, merge it into a cache, re-bundle our cache, and re-output. The only problem I can see is that bundles will get monotonically larger because, as dependencies get removed, they will remain in new bundles.

add option to override module identifier resolution functions

It was suggested in #43 that some users would like module identifier resolution to work differently than the node module resolution algorithm. This is perfectly acceptable according to the CommonJS specification, so let's add it.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1254371-add-option-to-override-module-identifier-resolution-functions?utm_campaign=plugin&utm_content=tracker%2F297897&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F297897&utm_medium=issues&utm_source=github).

-w option should console.log a message whenever it finishes rebuilding the bundle.

Currently when you run with -w you get a console log when it detects a change:

Rebuilding bundle starting at c:\NewProjects\glass\node\ui\webgl\Canvas.js

But it might take 4 or more seconds for it to complete and there is no message when it does.

It's important to get a message when the build completes so that you know when to reload your browser and test your changes.

Per file aliases

The current aliasing system does not allow for node style dependency resolution. i.e. if module A relies on "underscore v1.3.0" and module B relies on "underscore v1.4.4" I can only require one or the other.

I realize that there are obvious issues with excess file size when including multiple versions
of the same library to the browser, however, in most cases I'd rather have extra javascript than broken libraries.

My suggestion would be to allow aliases that only apply to a single file, i.e.

{ 
fileAliases: {
  "/components/module-a/index.coffee": { 
    "underscore": "/components/module-a/components/underscore/index.js" 
  },
  "/components/module-b/index.coffee": { 
    "underscore": "/components/module-b/components/underscore/index.js" 
  }
} 
}

Support input source maps when present

My project is authored in coffeescript (of course), and I build them with associated source.map files.

When using cjsify, it generates a source map back to the input javascript file, but does not use the .map as input.

Ideally, it should map all the way back to the original .coffee file.

Is future support for this planned, or maybe I am just missing some option.

add CLI tests

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1254339-add-cli-tests?utm_campaign=plugin&utm_content=tracker%2F297897&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F297897&utm_medium=issues&utm_source=github).

add --verbose flag

It should log to stderr information about dependency resolution and then information about required files.

core module symlinks are not working on Windows

Trying to build a trivial module on Windows that uses the 'assert' module fails:

c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\lib\index.js:418
throw e;
^
Error: Core module "c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\core\assert.js" has not yet been
ported to the browser
at resolvePathSync (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\lib\index.js:160:15)

CLI Option for Extension Handlers

It looks like the file watcher is only available on the CLI. The problem is that I need custom extension handler for my project.

Is there a way to add this to the CLI? I can't think of a great interface for it, but something like -h extension:path-to-file might be useful enough. That file would be a module that exports a function that acts just like the handlers object callbacks.

module.exports = (body, filename) ->
  body = body.toString()
  compiled = body.replace(/\"/g, "\\\"").replace(/\n/g, '\\\n')                                                                                                                 
  require('esprima').parse("module.exports = \"#{compiled}\";")

Then, if you allow lists in some way, I could use the same transform method for multiple extensions.

It seems like a complex option to pass on a command line, but it would be very useful to me.

npm test not working

It would be nice to get "npm test" to work correctly, and add the project to Travis CI, etc.
At the moment it fails because of the broken symlinks in core/ when run locally.

Did I get something wrong?

SourceMap file not found when changing file structure

Given file structure like:

➤➤ tree
.
|-- coffee
|   `-- main.coffee
|-- layout
|   `-- index.jade
`-- page
    |-- build.js
    |-- build.map
    `-- index.html

and I use this command to compile:

➤➤ cjsify -o page/build.js --source-map-file page/build.map -w coffee/main.coffee

(BTW, --source-map-file is so long to type, can we add a shortcut here?)
And I get sourceMappingURL like this:

//@ sourceMappingURL=page/build.map

which fails to load build.map since url is wrong.

This case could be bypasswd by putting both index.html build.js at project root directory where I run cjsify ....
I tried locally that sourceMappingURL is based on the path of .js file,
so I think sourceMappingURL may be inferences without adding a parameter.

-w option does not work on windows

Repro on Windows 7 with Nodjs 0.10.5:

Create file main.js

exports.hello = function(){
    console.log("hello");
}

run

cjsify main.js --output output.js -w

edit file main.js
result:

WARNING: watched file C:\NewProjects\cjsifybug\main.js has disappeared

output.js is also not updated.

When compiling coffee files there should be useful source file, line and column info

Also, I'm pretty sure the example code provided here should be valid coffeescript. This might be a bug in coffeescript-redux compiler.

Repro:
Create file error.coffee

fn = ->
    while true
        if true
            1
        else
            return 2

run cjsify error.coffee

Result:

C:\NewProjects\glass>cjsify error.coffee

c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\lib\command.js:96
        throw e;
              ^
Error: expr: Cannot use a ReturnStatement as a value
    at expr (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modules\coffee-script-redux\lib\com
piler.js:153:11)
    at expr (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modules\coffee-script-redux\lib\com
piler.js:114:14)
    at expr (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modules\coffee-script-redux\lib\com
piler.js:124:17)
    at expr (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modules\coffee-script-redux\lib\com
piler.js:135:52)
    at makeReturn (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modules\coffee-script-redux\l
ib\compiler.js:182:35)
    at class$.exports.Compiler.expression (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modul
es\coffee-script-redux\lib\compiler.js:1071:20)
    at class$.<anonymous> (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\node_modules\coffee-script
-redux\lib\compiler.js:2302:86)
    at class$.exports.Compiler.Compiler.compile.walk (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere
\node_modules\coffee-script-redux\lib\compiler.js:2215:19)
    at class$.exports.Compiler.Compiler.compile.walk (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere
\node_modules\coffee-script-redux\lib\compiler.js:2206:68)
    at class$.children.(anonymous function).child (c:\Users\Kris\AppData\Roaming\npm\node_modules\commonjs-everywhere\no
de_modules\coffee-script-redux\lib\compiler.js:2201:27)

Expected:

That sample should actually compile, or if it does fail, there should be file, line and column information at least, and ideally the relevant line text as well.

Note: This is on Windows 7 with Node v0.11.0

Performances

(from #11)

As you said, the tool is quite fast as is.

Build time for me :

cjsify : 1051ms
ls     : 220ms
hamlc  : 67ms
esprima: 37ms
AST->JS: 44ms
Total  : 1420ms
compiled script to wowboardhelpers.user.js

This is calculated here. The time spent in esprima, haml-coffee and livescript is not included in the csify step. AST->JS is done after.

`extensions` on suffixes instead of just extensions.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1254297-extensions-on-suffixes-instead-of-just-extensions?utm_campaign=plugin&utm_content=tracker%2F297897&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F297897&utm_medium=issues&utm_source=github).

SourceMap not working for core modules

I used events = require "events" in my code.
After sourceMap generated, I found there's a events file in my "Sources" in dev tools.
Same for the querystring module if I required it.
Clicking that events file, I got an 404 page of Nginx(I test my pages on Nginx).
Modules installed via npm install works right here.

I don't know if it's reasonable enough to add sourceMap for them, but it's an issue here to report.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.