ember-cli / broccoli-terser-sourcemap

Broccoli filter to uglify with sourcemaps

A broccoli filter that applies terser to minify code while properly generating or maintaining sourcemaps.

installation

npm install --save broccoli-terser-sourcemap

usage

const Terser = require('broccoli-terser-sourcemap');

// basic usage
let minified = new Terser(input);

// advanced usage
let minified = new Terser(input, {
  exclude: [..], // array of globs, to not minify

  terser: {
    mangle: false,    // defaults to true
    compress: false,  // defaults to true
    sourceMap: false, // defaults to true
    //...
  },

  publicUrl: 'https://myamazingapp.com/', // value to be prepended to sourceMappingURL, defaults to ''
  hiddenSourceMap: false, // skips adding the reference to sourcemap in the minified JS, defaults to false

  concurrency: 3 // number of parallel workers, defaults to number of CPUs - 1
});

To disable parallelization:

$ JOBS=0
$ JOBS=1

broccoli-terser-sourcemap's Issues

Wrong path in sourcemap sources entry for chunks in assets dir

ember-auto-import/webpack compiles vendor code into chunks in the assets directory in our project. Those files are then minimized using ember-cli-terser.

Since webpack is set to not build sourcemaps for those files, this plugin set the sources entry in the sourcemap to the relativePath of the file.

As an example, we get a sourcemap /assets/chunk.123.map with "sources": "assets/chunk.123.js". This leads to some issues where external tools try to read those maps, since they cannot find /assets/assets/chunk.123.js. Should it rather be "sources": "chunk.123.js" in this case since the path is relative and the map is already located at assets/?

I attempted a fix here: dagroe@5fe87d0

v2.0.0 gives error when running ember deploy

Error: The Broccoli Plugin: [Fingerprint] failed with:
Error: ENOENT: no such file or directory, open '/repo/tmp/uglify_writer-input_base_path-ZtW0omFx.tmp/assets/dag-map.js.map'
    at Object.fs.openSync (fs.js:653:18)
    at Object.fs.readFileSync (fs.js:554:33)
    at UglifyWriter.processFile (/repo/node_modules/broccoli-uglify-sourcemap/index.js:101:41)
    at /repo/node_modules/broccoli-uglify-sourcemap/index.js:65:16
    at Array.forEach (<anonymous>)
    at /repo/node_modules/broccoli-uglify-sourcemap/index.js:55:25
    at Array.forEach (<anonymous>)
    at UglifyWriter.build (/repo/node_modules/broccoli-uglify-sourcemap/index.js:54:19)
    at /repo/node_modules/broccoli-plugin/read_compat.js:93:34
    at tryCatch (/repo/node_modules/rsvp/dist/rsvp.js:525:12)
    at invokeCallback (/repo/node_modules/rsvp/dist/rsvp.js:538:13)
    at publish (/repo/node_modules/rsvp/dist/rsvp.js:508:7)
    at flush (/repo/node_modules/rsvp/dist/rsvp.js:2415:5)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)

I suspect this is because of the upgrade to source-map-url: ^0.4.0

Build failing with jquery-ui

Here is a reproduction repo: https://github.com/gdub22/sourcemap-jqueryui

Fresh ember-cli app, added jquery ui, enabled minification and sourcemaps: https://github.com/gdub22/sourcemap-jqueryui/commit/baafad281924290c9b84acf4302a77b35ef82a9f

Invalid mapping: {"generated":{"line":73687,"column":-50},"source":"bower_components/jquery-ui/jquery-ui.js","original":{"line":1,"column":0},"name":null}

Minification works fine if I disable sourcemaps.

Support for uglify-es

Now that Ember 2.13 has babel browser targeting, the build can contain ES6 code which uglify-js doesn't support. I've been using babili but it takes twice as long and produces larger files than uglify-es.

Someone has already made a branch for this but an official ember-cli packages would be great.

https://github.com/fusion2004/broccoli-uglify-es-sourcemap
https://github.com/fusion2004/ember-cli-uglify-es

UglifyJS 2.6.0 breaks broccoli-uglify-sourcemap

After this sub-dependency was updated to 2.6 yesterday, our app no longer builds in our production environment. Downgrading back down to UglifyJS 2.5.0 will resolve the issue. Not sure if there is a breaking change in that project that needs to be accommodated.

Error/stack trace:

Error received
Maximum call stack size exceeded
RangeError: Maximum call stack size exceeded
at AST_Node.eval [as transform](eval at %28/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0%29, <anonymous)
at eval (eval at (/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0), :3011:29)
at Object.merge.before (eval at (/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0), :5142:9)
at AST_Node.eval [as transform](eval at %28/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0%29, :2894:35)
at eval (eval at (/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0), :3012:29)
at Object.merge.before (eval at (/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0), :5142:9)
at AST_Node.eval [as transform](eval at %28/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0%29, :2894:35)
at eval (eval at (/Users/Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0), :3012:29)
at Object.merge.before (eval at (/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0), :5142:9)
at AST_Node.eval [as transform](eval at %28/Users//Dev//node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:1:0%29, :2894:35)

CHANGELOG outdated

The latest releases seem to be missing from the CHANGELOG.md file

/cc @mikrostew @stefanpenner

fix failing tests

looks like an unstable key ordering

release

changes since last: 0c1007b...master

could use another set of eyes & testers or two, this appears to work as expected (but better safe the sorry)

broccoli 1.0 support

thought why not create this issue and leave it here.

Cannot pass option to Uglify

I'm trying to pass the --mangle-props=unquoted option, yet it seems that Uglify may not be receiving this option from broccoli-uglify-sourcemap`, as the complied js file does not show the expected code (which keeps quoted object properties from being changed to the dot notation). Below is the sample project code I'm using:

// app/index.js
Foo['default'] = 'bar';

// Brocfile.js
var uglify        = require('broccoli-uglify-sourcemap');

module.exports =  uglify('app', {
  "mangle_props": "unquoted"
  /* 
  I also tried these variations:
  "mange-props":  "unquoted",
  "mangeProps":   "unquoted",
  "mange_props":  2, // 2 == the option for unquoted
  "mange-props":  2,
  "mangeProps":   2
  "keep_quoted_props": true
  "mangleProperties": {
    "ignore_quoted": true
  }
  */
});

// the resulting compiled dist/index.js
Foo.default="bar";
//# sourceMappingURL=index.map

I wonder if this issue is with broccoli-uglify-sourcemap, because when I use Uglify directly in the command line, I get the expected results:

$ uglifyjs app/index.js --compress --mangle-props=unquoted
> Foo["default"]="bar";

I'm not certain if the issue is with broccoli-uglify-sourcemap or with the uglify-js npm package. Is there a different way that options should be passed to Uglify?

Alternately (to solve my current issue), is there a way to have broccoli-uglify-sourcemap instruct Uglify to prevent using the dot notation for any quoted object properties that are reserved JS words (default, with, for, etc.)? Using the dot notation with these reserved words is causing our application to fail in IE8 (which we unfortunately still have to support).

`EISDIR: illegal operation on a directory, read` error after library upgrade

Hello!

We recently upgraded @auth0/auth0-spa-js to a more recent version and our production build started failing with this error:

cleaning up...
Build Error (UglifyWriter)
EISDIR: illegal operation on a directory, read

Notably v1.6.2 does not have this error, but anything from 1.7.0 does.

Reproducing it locally I found this log from the error dump:

ERROR Summary:

  - broccoliBuilderErrorStack: Error: EISDIR: illegal operation on a directory, read
    at Object.readSync (fs.js:523:3)
    at tryReadSync (fs.js:348:20)
    at Object.readFileSync (fs.js:385:19)
    at getSourceMapContent (/Users/meirish/code/cloud-ui/node_modules/broccoli-uglify-sourcemap/lib/get-sourcemap-content.js:18:30)
    at processFile (/Users/meirish/code/cloud-ui/node_modules/broccoli-uglify-sourcemap/lib/process-file.js:41:19)
    at Function.processFileParallel (/Users/meirish/code/cloud-ui/node_modules/broccoli-uglify-sourcemap/lib/worker.js:9:10)
    at MessagePort.<anonymous> (/Users/meirish/code/cloud-ui/node_modules/broccoli-uglify-sourcemap/node_modules/workerpool/dist/workerpool.min.js:25:16559)
    at MessagePort.emit (events.js:311:20)
    at MessagePort.onmessage (internal/worker/io.js:78:8)

What ends up happening here is srcURL.from() here: https://github.com/ember-cli/broccoli-uglify-sourcemap/blob/master/lib/get-sourcemap-content.js#L10 returns an empty string '' as the first item in the urls array when it gets to our app's vendor.js file, which then errors here: https://github.com/ember-cli/broccoli-uglify-sourcemap/blob/master/lib/get-sourcemap-content.js#L18 when it tries to read a directory instead of a file.

Removing the empty string from the array results in a successful build, but with this warning:

[UglifyWriter][WARN] (broccoli-uglify-sourcemap) "chunk.40bafbb6f4ed25560db2.js.map;", "chunk.cc45f8eb6b4af1864c92.js.map//#" referenced in "assets/vendor.js" could not be found

Which seems to point to ember-auto-import.

I'm uncertain where the issue is - is it in the auth0 source map urls, the source-map-url package that returns '', or maybe ember-auto-import. Has anyone ever seen this or have any guidance for where to look further?

Move to terser breaks the order of yield in generators

Changes in 4ca908a replaced the uglify-es dependency with terser. This has a bug with generators (terser/terser#60) which has broken our use of wrap from ember-co.

As this is a minor version update it is picked up by our dependency on [email protected]

enable CI

tests appear to be failing

will investigate shortly

parallelize processFile

https://github.com/ember-cli/broccoli-uglify-sourcemap/blob/master/index.js#L65

Allow slow concat time to be configurable

This is related to #36 #38. Currently, the only way to suppress the message:

[WARN] (broccoli-uglify-sourcemap) Minifying: 'assets/somefile.js' took: 21087ms (more than 20,000ms)

is by using the --silent option, which also suppresses some other messages which I don't want to suppress. I propose that we allow the "slow" concat time to be configurable (it is hard coded at 20000ms right now). Then users can suppress this message only by increasing the time.

improve support of malformed/missing/broken sourcemaps

transplanted from : ember-cli/ember-cli-terser#4 (comment)

---

@stefanpenner

Here's an updated repo which reproduces the issue: https://github.com/johnnyshields/ember-sourcemap-issue

I believe I found the problem: ember-cli-uglify requires 3rd-party bower libs to have their magic sourcemap at the BEGINNING of their source, so that Ember CLI can concatenate them like this:

//# sourceMappingURL=lib1.map
lib1.js code

lib2.js code   // this lib doesn't have a sourcemap

//# sourceMappingURL=lib3.map
lib3.js code

If a lib declares the magic comment at the END of its file, the following happens:

lib1.js code

//# sourceMappingURL=lib1.map      ember gets con
lib2.js code

Because Ember CLI evaluates the maps on the CONCATENATED file, here it apparently gets confused and thinks the source map is for lib2 when it's actually for lib1.

So, infact in the original post of this thread, select2 is not the issue--whichever JS comes BEFORE select2 is the issue, but error thrown looks like it's select2.

My sample repo uses sprintf.js as an offender which has magic comment at the end. If you manually edit the sprintf.js to have the magic comment at the beginning, everything works.

Circle CI Workers are terminated during build

Hi, recently I had an issue with failing build on CircleCI.
It wasn't happening on ember 2.8 and ember-cli-uglify 2.1.0.
Currently we have ember 3.14 and ember-cli-terser 4.0.1

ember build -e=production

Workerpool Worker terminated Unexpectedly
    exitCode: `null`
    signalCode: `SIGKILL`
    workerpool.script: `/home/circleci/project/node_modules/broccoli-terser-sourcemap/lib/worker.js`
    spawnArgs: `/usr/local/bin/node,/home/circleci/project/node_modules/broccoli-terser-sourcemap/lib/worker.js`
    spawnfile: `/usr/local/bin/node`
    stdout: `null`
    stderr: `null`

What I did to fix the issue and make build green again was JOBS=1 ember build -e=production which sets worker pool to 1.
The issue might be related to CircleCi showing available cores of the whole machine rather than the used instance.

Although the concurrency flag logic haven't changed since 2.1.0.

I'm creating this issue mostly for discoverability so others have easier time finding out a possible fix.

Feel free to close this.
I can also add info to this and ember-cli-terser repo's readme.

Console spam when parsing inline sourcemaps

Problem

When this addon encounters an inline sourcemap (sourceMappingURL=data:application/json;charset=utf-8;base64,[giant base64 blob...]), it fails to parse it and generates a huge console message containing the entire base64 blob.

Cause

The source-map-url package used in this lib will return the entire inline sourcemap when you call getFrom(code).

This becomes an issue in get-sourcemap-content.js, because it always assumes getFrom(code) returns a url.

The filesystem exists check fails, and a console.warn message fires off saying the url referenced could not be found. However this url is actually the huge inline sourcemap blob!

New dependency name: uglify-es => terser #1

There has been development on a new fork of uglify-es, which is now called terser and has its own package!

"terser": "^3.7.3"

Please upgrade to our new version.

Cannot override mangle and compress defaults

It appears that mangle and compress options are true by default and can't be overriden. I'd like to turn them off in certain circumstances (when speed of processing is a premium over size of assets). What was the reasoning behind having these options always turned on?

production builds failing in ember 3.0.0.beta.2: broccoliBuilderErrorStack: TypeError: name.definition is not a function #35

production builds started failing when I upgraded to ember 3.0.0.beta.2

ember build -e production
cleaning up...
Build failed.
Build Error (UglifyWriter)

name.definition is not a function


Stack Trace and Error Report: /var/folders/ss/sjsdtptn7j1cc6y_x8lbc3ch0000gnT/error.dump.84a92b54f2b31e423fbac3414a0600c9.log

/var/folders/ss/sjsdtptn7j1cc6y_x8lbc3ch0000gn/T/error.dump.84a92b54f2b31e423fbac3414a0600c9.log

=================================================================================

ENV Summary:

  TIME: Sat Feb 03 2018 13:15:30 GMT-0500 (EST)
  TITLE: ember
  ARGV:
  - /Users/x/.nvm/versions/node/v8.9.3/bin/node
  - /usr/local/bin/ember
  - build
  - -e
  - production
  EXEC_PATH: /Users/ron/.nvm/versions/node/v8.9.3/bin/node
  TMPDIR: /var/folders/ss/sjsdtptn7j1cc6y_x8lbc3ch0000gn/T
  SHELL: /usr/local/bin/bash
  PATH:
  - /Users/x/.pyenv/shims
  - /Users/x/.rvm/gems/ruby-head/bin
  - /Users/x/.rvm/gems/ruby-head@global/bin
  - /Users/x/.rvm/rubies/ruby-head/bin
  - /Users/x/.nvm/versions/node/lts/*/bin
  - /Users/x/.nvm/versions/node/v8.9.3/bin
  - /usr/local/bin
  - /usr/bin
  - /bin
  - /usr/sbin
  - /sbin
  - /usr/local/MacGPG2/bin
  - /Users/x/bin
  - /Applications/Xcode.app/Contents/Developer/usr/bin
  - /Users/x/.rvm/bin
  - /Users/x/.config/yarn/global/node_modules/.bin
  PLATFORM: darwin x64
  FREEMEM: 386052096
  TOTALMEM: 17179869184
  UPTIME: 231154
  LOADAVG: 3.248046875,2.896484375,2.8720703125
  CPUS:
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  - Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz - 2000
  ENDIANNESS: LE
  VERSIONS:
  - ares: 1.10.1-DEV
  - cldr: 31.0.1
  - http_parser: 2.7.0
  - icu: 59.1
  - modules: 57
  - nghttp2: 1.25.0
  - node: 8.9.3
  - openssl: 1.0.2n
  - tz: 2017b
  - unicode: 9.0
  - uv: 1.15.0
  - v8: 6.1.534.48
  - zlib: 1.2.11

ERROR Summary:

  - broccoliBuilderErrorStack: TypeError: name.definition is not a function
    at append_var (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11923:28)
    at flatten_vars (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11971:21)
    at flatten_fn (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11990:13)
    at eval (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11792:44)
    at AST_Call.eval [as optimize] (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:7666:23)
    at Compressor.before (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:7649:24)
    at AST_Call.eval [as transform] (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:4559:35)
    at eval (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:4618:49)
    at Compressor.before (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:7644:9)
    at AST_Return.eval [as transform] (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:4559:35)
  - codeFrame: name.definition is not a function
  - errorMessage: Build Canceled: Broccoli Builder ran into an error with `UglifyWriter` plugin. 💥
name.definition is not a function
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: [undefined]
    - line: [undefined]
    - treeDir: [undefined]
  - message: Build Canceled: Broccoli Builder ran into an error with `UglifyWriter` plugin. 💥
name.definition is not a function
  - name: Error
  - nodeAnnotation: UglifyWriter
  - nodeName: UglifyWriter
  - originalErrorMessage: name.definition is not a function
  - stack: TypeError: name.definition is not a function
    at append_var (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11923:28)
    at flatten_vars (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11971:21)
    at flatten_fn (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11990:13)
    at eval (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:11792:44)
    at AST_Call.eval [as optimize] (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:7666:23)
    at Compressor.before (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:7649:24)
    at AST_Call.eval [as transform] (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:4559:35)
    at eval (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:4618:49)
    at Compressor.before (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:7644:9)
    at AST_Return.eval [as transform] (eval at <anonymous> (/Users/x/mirai-audio/mir/node_modules/uglify-es/tools/node.js:21:1), <anonymous>:4559:35)

=================================================================================

I narrowed it down to ember-cli-uglify by disabling the addon in my ember-cli-build.js file, at which point builds would succeed.

ember-cli-build.js

module.exports = function(defaults) {
  let app = new EmberApp(defaults, {
    // ...
    'ember-cli-uglify': {
      uglify: {
        compress: false // added this to workaround the issue
      }
    },
    // ...

more details:

$ember --version --verbose

ember-cli: 3.0.0-beta.2
http_parser: 2.7.0
node: 8.9.3
v8: 6.1.534.48
uv: 1.15.0
zlib: 1.2.11
ares: 1.10.1-DEV
modules: 57
nghttp2: 1.25.0
openssl: 1.0.2n
icu: 59.1
unicode: 9.0
cldr: 31.0.1
tz: 2017b
os: darwin x6

A repo demonstrating the failure is here at this commit:
https://github.com/mirai-audio/mir/tree/7bc4b65df5d01e085c98122c8c8a79542a26dad9

Switch from uglify-es to terser

uglify-es has been unmaintained for quite a while now. I've been following terser for a while now and it has now been officially renamed to that and has an official package in npm. It is continued by two of the biggest contributors of uglify-es. I suggest using terser instead of uglify-es from now on. And a new release including it would be nice. :)

Ember source map failing

Hi guys,
last days I was struggling with failing build process with sourcemaps by Invalid mapping stuff. I found out, that 16 days ago new version of uglify-js was released (2.2.17).

So I tried in package.json set version of this package exactly to previous version 2.4.16 (from "uglify-js": "^2.4.16" to "uglify-js": "2.4.16") and everything works well again.

I don't know what is wrong with new version of uglify-js. Could you change it also in your repository till the issue with uglify-js will be solved?

WARNING: Option "nodeWorker" is deprecated since [email protected]. Please use "workerType" instead.

Option "nodeWorker" is used: https://github.com/ember-cli/broccoli-uglify-sourcemap/blob/v3.2.0/index.js#L53
Dependency on workerpool 5.+: https://github.com/ember-cli/broccoli-uglify-sourcemap/blob/v3.2.0/package.json#L37

WARNING: Option "nodeWorker" is deprecated since [email protected]. Please use "workerType" instead.

This project appears to introduce ~1131 files via itself and its deps

own files mistakenly published:

• test 19

dependencies to investigate:

• broccoli-plugin 324 broccolijs/broccoli-plugin#13
• debug 20
• lodash 434 (often we can use merely a subset of lodash, via one of the subset modules such as lodash.uniq. we should explore that)
• matcher-collection 35
• mkdirp 45
• source-map-url 12
• symlink-or-copy 11
• uglify-js 193
• walk-sync 20

should this be called broccoli-terser-sourcemap?

Since we moved away from uglify and onto the terser fork it was confusing to see what to debug as uglify is not being used.

Implement a better way of comparing mapfile output

The current way we check the output of sourcemap files is fragile. It has broken in the past when uglify js changes the output order of their map.

We should implement a solution that is more stable (deep comparison of expected output) to make the tests more tolerant of ordering changes.

"<URL> referenced in <RELATIVE_PATH> could not be found error" upgrading to v4

Hi!
I recently upgraded from ember-cli-uglify@3 to ember-cli-terser@4 and now I'm seeing an error related to sourcemap URL's when generating production build with sourcemaps enabled.
This results in sourcemaps referenced as to a relative path //# sourceMappingURL=app-872148ced72309772aa624ced3dfb68b.map instead of absolute //# sourceMappingURL=https://s3-eu-central-1.amazonaws.com/blabla/assets/app-872148ced72309772aa624ced3dfb68b.map.

No additional ember-cli-terser configuration is set and exactly the same configuration used to work in ember-cli-uglify. Seems like wrong hash is being applied to .map files.

Console output

[TerserWriter][WARN] (broccoli-terser-sourcemap) "https://s3-eu-central-1.amazonaws.com/blabla/assets/app-a7d016737320d4b33763a690926b622f.map" referenced in "assets/app-872148ced72309772aa624ced3dfb68b.js" could not be found
[WARN] (broccoli-terser-sourcemap) "https://s3-eu-central-1.amazonaws.com/blabla/assets/vendor-859e1305b1dc366a97b803eec97b4df5.map" referenced in "assets/vendor-b3eeea38d69cb5b99f35ff8805ab60a3.js" could not be found

Configuration

    fingerprint: {
      enabled: true,
      extensions: [
        "pdf",
        "js",
        "css",
        "png",
        "jpg",
        "gif",
        "map",
        "otf",
        "eot",
        "svg",
        "ttf",
        "woff",
        "woff2",
      ],
      prepend: "https://s3-eu-central-1.amazonaws.com/blabla/",
    },

    sourcemaps: {
      enabled: true,
      extensions: ["js"],
    },

Dist folder contents

• app-872148ced72309772aa624ced3dfb68b.js
• app-872148ced72309772aa624ced3dfb68b.map
• vendor-4d134e6c58b511425ff083b735069361.css
• vendor-b3eeea38d69cb5b99f35ff8805ab60a3.js
• vendor-b3eeea38d69cb5b99f35ff8805ab60a3.map

prepend portion of asset-urls not accounted for

Related ember-cli/ember-cli-terser#275 (comment)

When specifying

new EmberApp(defaults, {
  fingerprint: {
    prepend: '<<CDN URL>>'
  }
})

It appears to be impossible to build with ember-cli-terser enabled at all, even when deactivating sourcemaps via

new EmberApp(defaults, {
  sourcemaps: {
    enabled: false
  }
})

Just in case ember-cli-terser and broccoli-terser-sourcemap need to be aware of this URL I added the publicUrl option

new EmberApp(defaults, {
  'ember-cli-terser': {
    publicUrl: '<<CDN URL>>'
  },
  fingerprint: {
    enabled: false,
    publicUrl: '<<CDN URL>>'
  }
})

All of this was to no avail. Even with sourcemaps disabled I get logs like the following:

[WARN] (broccoli-terser-sourcemap) "<<CDN URL>>/assets/leaflet/leaflet-src-18322c41aab5808cecb4784380d74637.js.map" referenced in "assets/leaflet/leaflet-src-18322c41aab5808cecb4784380d74637.js" could not be found

As well as a build error:

Unexpected token punc «:», expected punc «,»

Using a debugger I find that this error is from within a sourcemap being parsed (surprising with them disabled?!)

Update to terser v4

v4.0.0 of terser has been released. It would be nice to see this PR merged and released.

ember-cli build fails when enabling sourcemap

Here are the steps to reproduce:

(Using ember-cli 0.2.0)

$ ember new foo
  ... output ...
$ cd foo

Edit Brocfile.js

var app = new EmberApp({
  sourcemaps: ['js']
});

Build it

$ ember build --env production

The output of the last step

Kamals-MacBook-Pro:baz kamal$ ember build --env production
The package `ember-data` is not a properly formatted package, we have used a fallback lookup to resolve it at `/Users/kamal/src/baz/node_modules/ember-data`. This is generally caused by an addon not having a `main` entry point (or `index.js`).
version: 0.2.0
BuildingThe package `ember-data` is not a properly formatted package, we have used a fallback lookup to resolve it at `/Users/kamal/src/baz/node_modules/ember-data`. This is generally caused by an addon not having a `main` entry point (or `index.js`).
Build failed.
Invalid mapping: {"generated":{"line":74919,"column":-50},"source":"vendor/ember-cli/vendor-suffix.js","original":{"line":1,"column":0},"name":null}
Error: Invalid mapping: {"generated":{"line":74919,"column":-50},"source":"vendor/ember-cli/vendor-suffix.js","original":{"line":1,"column":0},"name":null}
    at SourceMapGenerator_validateMapping [as _validateMapping] (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/node_modules/source-map/lib/source-map/source-map-generator.js:272:15)
    at SourceMapGenerator_addMapping [as addMapping] (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/node_modules/source-map/lib/source-map/source-map-generator.js:102:12)
    at /Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/node_modules/source-map/lib/source-map/source-map-generator.js:74:19
    at Array.forEach (native)
    at SourceMapConsumer_eachMapping [as eachMapping] (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/node_modules/source-map/lib/source-map/source-map-consumer.js:473:10)
    at Function.SourceMapGenerator_fromSourceMap (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/node_modules/source-map/lib/source-map/source-map-generator.js:50:26)
    at Object.SourceMap (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/lib/sourcemap.js:59:52)
    at Object.exports.minify (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/node_modules/uglify-js/tools/node.js:112:38)
    at UglifyWriter.processFile (/Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/index.js:98:25)
    at /Users/kamal/src/baz/node_modules/ember-cli-uglify/node_modules/broccoli-uglify-sourcemap/index.js:46:14

Broken with Unicode Regex

We upgraded to ember-cli-uglify 2.0.0 (from 1.2.0) and found that our production builds were broken. So debugging pointed to unicode regex from a library we're using. The library is emojione.

Here is the original regex in source:

regUnicode = new RegExp("<object[^>]*>.*?<\/object>|<span[^>]*>.*?<\/span>|<(?:object|embed|svg|img|div|span|p|a)[^>]*>|(?:\uD83C\uDFF3)\uFE0F?\u200D?(?:\uD83C\uDF08)|(?:\uD83D\uDC41)\uFE0F?\u200D?(?:\uD83D\uDDE8)\uFE0F?|[#-9]\uFE0F?\u20E3|(?:(?:\uD83C\uDFF4)(?:\uDB40[\uDC60-\uDCFF]){1,6})|(?:\uD83C[\uDDE0-\uDDFF]){2}|(?:(?:\uD83D[\uDC68\uDC69]))\uFE0F?(?:\uD83C[\uDFFA-\uDFFF])?\u200D?(?:[\u2695\u2696\u2708]|\uD83C[\uDF3E-\uDFED]|\uD83D[\uDCBB\uDCBC\uDD27\uDD2C\uDE80\uDE92])|(?:\uD83D[\uDC68\uDC69]|\uD83E[\uDDD0-\uDDDF])(?:\uD83C[\uDFFA-\uDFFF])?\u200D?[\u2640\u2642\u2695\u2696\u2708]?\uFE0F?|(?:(?:\u2764|\uD83D[\uDC66-\uDC69\uDC8B])[\u200D\uFE0F]{0,2}){1,3}(?:\u2764|\uD83D[\uDC66-\uDC69\uDC8B])|(?:(?:\u2764|\uD83D[\uDC66-\uDC69\uDC8B])\uFE0F?){2,4}|(?:\uD83D[\uDC68\uDC69\uDC6E\uDC71-\uDC87\uDD75\uDE45-\uDE4E]|\uD83E[\uDD26\uDD37]|\uD83C[\uDFC3-\uDFCC]|\uD83E[\uDD38-\uDD3E]|\uD83D[\uDEA3-\uDEB6]|\u26f9|\uD83D\uDC6F)\uFE0F?(?:\uD83C[\uDFFB-\uDFFF])?\u200D?[\u2640\u2642]?\uFE0F?|(?:[\u261D\u26F9\u270A-\u270D]|\uD83C[\uDF85-\uDFCC]|\uD83D[\uDC42-\uDCAA\uDD74-\uDD96\uDE45-\uDE4F\uDEA3-\uDECC]|\uD83E[\uDD18-\uDD3E])\uFE0F?(?:\uD83C[\uDFFB-\uDFFF])|(?:[\u2194-\u2199\u21a9-\u21aa]\uFE0F?|[\u0023\u002a]|[\u3030\u303d]\uFE0F?|(?:\ud83c[\udd70-\udd71]|\ud83c\udd8e|\ud83c[\udd91-\udd9a])\uFE0F?|\u24c2\uFE0F?|[\u3297\u3299]\uFE0F?|(?:\ud83c[\ude01-\ude02]|\ud83c\ude1a|\ud83c\ude2f|\ud83c[\ude32-\ude3a]|\ud83c[\ude50-\ude51])\uFE0F?|[\u203c\u2049]\uFE0F?|[\u25aa-\u25ab\u25b6\u25c0\u25fb-\u25fe]\uFE0F?|[\u00a9\u00ae]\uFE0F?|[\u2122\u2139]\uFE0F?|\ud83c\udc04\uFE0F?|[\u2b05-\u2b07\u2b1b-\u2b1c\u2b50\u2b55]\uFE0F?|[\u231a-\u231b\u2328\u23cf\u23e9-\u23f3\u23f8-\u23fa]\uFE0F?|\ud83c\udccf|[\u2934\u2935]\uFE0F?)|[\u2700-\u27bf]\uFE0F?|[\ud800-\udbff][\udc00-\udfff]\uFE0F?|[\u2600-\u26FF]\uFE0F?|[\u0030-\u0039]\uFE0F", "g");

Here is what it is uglify'd to:

e.regUnicode = new RegExp("<object[^>]*>.*?</object>|<span[^>]*>.*?</span>|<(?:object|embed|svg|img|div|span|p|a)[^>]*>|(?:🏳)️?‍?(?:🌈)|(?:👁)️?‍?(?:🗨)️?|[#-9]️?⃣|(?:(?:🏴)(?:�[�-�]){1,6})|(?:�[�-�]){2}|(?:(?:�[��]))️?(?:�[�-�])?‍?(?:[⚕⚖✈]|�[�-�]|�[������])|(?:�[��]|�[�-�])(?:�[�-�])?‍?[♀♂⚕⚖✈]?️?|(?:(?:❤|�[�-��])[‍️]{0,2}){1,3}(?:❤|�[�-��])|(?:(?:❤|�[�-��])️?){2,4}|(?:�[����-���-�]|�[��]|�[�-�]|�[�-�]|�[�-�]|⛹|👯)️?(?:�[�-�])?‍?[♀♂]?️?|(?:[☝⛹✊-✍]|�[�-�]|�[�-��-��-��-�]|�[�-�])️?(?:�[�-�])|(?:[↔-↙↩-↪]️?|[#*]|[〰〽]️?|(?:�[�-�]|🆎|�[�-�])️?|Ⓜ️?|[㊗㊙]️?|(?:�[�-�]|🈚|🈯|�[�-�]|�[�-�])️?|[‼⁉]️?|[▪-▫▶◀◻-◾]️?|[©®]️?|[™ℹ]️?|🀄️?|[⬅-⬇⬛-⬜⭐⭕]️?|[⌚-⌛⌨⏏⏩-⏳⏸-⏺]️?|🃏|[⤴⤵]️?)|[✀-➿]️?|[�-�][�-�]️?|[☀-⛿]️?|[0-9]️","g"),

It sees the problem is that the unicode chars are being converted to their escaped values. I will work to create and ember app that serves as a reproduction.

Update to terser v5

v5.0.0 of terser has been released.

This is important as 1) it's good to use the latest version of the production minifier and 2) nullish coalescing / optional chaining support will only be released in terser 5 which is needed to fix this other issue in ember-cli.

Document the options

Which options are supported ? There is no documentation about that.
I want to pass an option to uglify to not remove comments. How can I do it ?

Regards

Time based warning is not specific enough to identify source.

This warning does not give enough context to know what is actually causing the output.

Seeing the following lines in my CI logs (surrounded by a thousand lines of other things) doesn't provide enough context:

[WARN] `assets/web-client.js` took: 30636ms (more than 20,000ms)
[WARN] `assets/web-client.js` took: 22009ms (more than 20,000ms)

---

I suggest that we update the warning to at least say something like:

[WARN] (broccoli-uglify-sourcemap) Minifying assets/web-client.js took: XXXXXms

Thoughts?

Support publicPath and hidden-source-map

webpack supports a variety of options for source maps, which we could bring to the Ember community as well. Two of them that I feel can be useful are:

publicPath: The value will be prepended to the value of sourceMappingURL. So, for a value of https://my-vpn/, the generated sourceMappingURL will be, //# sourceMappingURL=https://my-vpn/app.map. Implementing this should be as simple as prepending the value of publicPath to url here.

hidden-source-map: Using this option will prevent the reference to the source map from being added to the minified asset. The source maps can be separately uploaded to a private server and then loaded using the "Add Source Map..." option in Chrome. To implement this, we would just have to stop passing the url option to terser.

Further Reading

There’s an I missing in the description!

...and it should be filter instead of fitler

sources:["0"] in `no-upstream` and `with-broken` sourcemap

In https://github.com/ember-cli/broccoli-uglify-sourcemap/pull/55/files#r263930016, broccoli-uglify-sourcemap would output sources\":[\"0\"], is there a way we can get the correct sources list here?

console.warn does not abide by --silent

console.warn should be removed, or migrated to heimdalljs-logger, or maybe abide by --silent
https://github.com/ef4/broccoli-uglify-sourcemap/blob/master/index.js#L131

Also, this warning does not provide sufficient context.