microsoft / parallel-prettier

Concurrent prettier runner

@mixer/parallel-prettier

A wrapper around prettier that formats files in parallel to speed up large projects.

npm install -g @mixer/parallel-prettier

After installing the CLI, the pprettier command will be available to you.

Ad-hoc Performance

# 300 file project:

prettier --write "src/**/*.{ts,tsx,json,scss}"  6.57s user 0.55s system 131% cpu 5.405 total
pprettier --write "src/**/*.{ts,tsx,json,scss}"  0.41s user 0.08s system 14% cpu 3.455 total

# 1200 file project:

prettier --write "src/**/*.{ts,tsx,json,scss}"  27.09s user 3.26s system 123% cpu 24.496 total
pprettier --write "src/**/*.{ts,tsx,json,scss}"  1.21s user 0.27s system 14% cpu 10.580 total

parallel-prettier's Issues

Prettier v3 support (and exceptions)

Hi, I get following error when I try to use pprettier with Prettier v3:

Error: unknown type: "ChainExpression"
    at printPathNoParens (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:30038:17)
    at Object.genericPrint [as print] (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:29629:23)
    at callPluginPrintFunction (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8601:26)
    at mainPrintInternal (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8550:22)
    at /Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8542:32
    at AstPath.call (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8359:24)
    at mainPrint (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8542:21)
    at printPathNoParens (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:29718:19)
    at Object.genericPrint [as print] (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:29629:23)
    at callPluginPrintFunction (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8601:26)

and

Error: unknown node type: Script
    at Object.print (/Users/.../node_modules/prettier-plugin-svelte/plugin.js:1430:11)
    at callPluginPrintFunction (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8601:26)
    at mainPrintInternal (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8550:22)
    at mainPrint (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8537:18)
    at AstPath.call (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8359:24)
    at printTopLevelParts (/Users/.../node_modules/prettier-plugin-svelte/plugin.js:1471:33)
    at Object.print (/Users/.../node_modules/prettier-plugin-svelte/plugin.js:892:16)
    at callPluginPrintFunction (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8601:26)
    at mainPrintInternal (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8550:22)
    at mainPrint (/Users/.../node_modules/@mixer/parallel-prettier/node_modules/prettier/index.js:8537:18)

Do you plan to update the library to support v3?

Reproduce the issue

git clone https://github.com/dfinity/nns-dapp
cd nns-dapp/frontend
git checkout build/prettier-v3
npm ci
npm run format

No performance gain

There is almost no performance improvement with --check. Tested using hyperfine, running on 32 cores.

> hyperfine --runs 2 'pprettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"' 'prettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"'
Benchmark #1: pprettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"
  Time (mean ± σ):     30.887 s ±  0.266 s    [User: 41.467 s, System: 5.283 s]
  Range (min … max):   30.699 s … 31.075 s    2 runs

Benchmark #2: prettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"
  Time (mean ± σ):     35.826 s ±  0.070 s    [User: 56.416 s, System: 2.453 s]
  Range (min … max):   35.777 s … 35.875 s    2 runs

Summary
  'pprettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"' ran
    1.16 ± 0.01 times faster than 'prettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"'

@mixer/parallel-prettier version 2.0.3 / prettier version 2.8.3

Prettier error traps parallel-prettier in infinite loop

I'm using this package to format a folder of generated markup. However, if the markup contains an error, for example an opening tag which is not closed

<div class="myClass">
  <div class="myClass_inner">
  <a href="#">Link</a>
</div>

it would run forever if not stopped manually. The original prettier throws an error, continues to format the other files and returns an exit code > 0.

In my mind, this should be an easy fix, since prettier already spits out the error/exit code, however I don't have much experience with gulp in general.

I'd be happy to provide a test repo or try to fix this issue if I can get some pointers in the right direction.

Support --cache / --cache-strategy options

Prettier v2.7.0 just shipped with support for --cache and --cache-strategy options that allow prettier to skip doing work on files that have not changed.

It'd be great if parallel-prettier could support the same options and cache file to avoid work where possible and make it even faster.

Use peerDependencies for prettier

So developers can bring their favorite version of prettier.

Changes:

• Move prettier@^1.17.0 to devDependencies
• Add prettier@1 to peerDependencies

MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 message listeners added to [Worker]

I noticed this issue while running parallel-prettier on ~16000 files in aws/aws-sdk-js-v3#1949

System Information

• Operating System: Amazon Linux 2 x86_64
• Host type: c5.4xlarge (16 Virtual CPUs, 32.0 GiB Memory, 160 GiB SSD (EBS) Storage)
• Node.js version: v14.15.4

Command run was equivalent to:

node --trace-warnings ./node_modules/.bin/pprettier --write **/*.{ts,js,md,json}

(node:697) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 message listeners added to [Worker]. Use emitter.setMaxListeners() to increase limit
    at _addListener (events.js:390:17)
    at Worker.addListener (events.js:406:10)
    at setupSubscription (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/observable/fromEvent.js:38:19)
    at Observable._subscribe (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/observable/fromEvent.js:25:9)
    at Observable._trySubscribe (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/Observable.js:44:25)
    at Observable.subscribe (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/Observable.js:30:22)
    at MapOperator.call (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/operators/map.js:32:23)
    at Observable.subscribe (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/Observable.js:25:31)
    at FilterOperator.call (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/operators/filter.js:29:23)
    at Observable.subscribe (/local/home/trivikr/workspace/aws-sdk-js-v3/node_modules/rxjs/internal/Observable.js:25:31)

The error appears around 16 times, equal to number of virtual CPUs.

Don't pin the Prettier version

Thanks for this awesome tool! On my 8-core machine, it reduces the test time from 5s to 0.6s.

This package brings Prettier as its own dependency. What do you think about making Prettier a peer dependency, similar to how glob-promise does it with glob? This way, everybody can use your tool with the version of Prettier that they have on their system, and you don't have to push updates for parallel-prettier each time Prettier releases a new version.

Support typescript (`--parser` argument)

It looks like the --parser argument is missing so TypeScript projects can't use this?

support editorconfig

prettier reads and honors editorconfig: https://prettier.io/docs/en/api.html#prettierresolveconfigfilepath--options

pprettier doesn't seem to.

So if you have in your .editorconfig:

[*.{js,ts,jsx,tsx}]
max_line_length = 100

then prettier will set printWidth to 100 but pprettier will keep it at the default 80.

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Inherit .gitignore

Prettier inherits .gitignore where as parrallel-prettier doesn't

I'm happy to make a PR but how do you want to achieve it? allow multiple ignore paths? include it if it exists?

"path must not be empty" error when used with "." as path

Supported by prettier (makes sense when used with a .prettierignore file)
https://prettier.io/docs/en/cli.html
the "." path option causes an error when used with this parallel prettier.

$ pprettier --check .
⠋ Starting........@mixer/parallel-prettier/node_modules/rxjs/internal/util/hostReportError.js:4
    setTimeout(function () { throw err; }, 0);
                             ^

TypeError: path must not be empty
    at throwError (.....@mixer/parallel-prettier/node_modules/ignore/index.js:366:9)
    at checkPath (.....@mixer/parallel-prettier/node_modules/ignore/index.js:379:12)
    at Ignore._test (.....@mixer/parallel-prettier/node_modules/ignore/index.js:502:5)
    at Ignore.ignores (.....@mixer/parallel-prettier/node_modules/ignore/index.js:541:17)
    at Pumpify.<anonymous> (.....@mixer/parallel-prettier/dist/master.js:29:25)
    at Pumpify.emit (events.js:314:20)
    at addChunk (.....@mixer/parallel-prettier/node_modules/readable-stream/lib/_stream_readable.js:291:12)
    at readableAddChunk (.....@mixer/parallel-prettier/node_modules/readable-stream/lib/_stream_readable.js:278:11)
    at Pumpify.Readable.push (.....@mixer/parallel-prettier/node_modules/readable-stream/lib/_stream_readable.js:245:10)
    at Pumpify.Duplexify._forward (.....@mixer/parallel-prettier/node_modules/duplexify/index.js:170:26)

Not respecting `.prettierignore`

pprettier does not seem to ignore node_modules by default, but Prettier does that, see https://prettier.io/docs/en/ignore.html.

@mixer/parallel-prettier version 2.0.3 / prettier version 2.8.3

Example output when not having added node_modules to .prettierignore

> pprettier --check "**/*.{css,html,js,json,jsonc,ts,yaml}"
⠸ Starting...
Error: Cannot find module '@ajv-validator/config/prettierrc.json'
Require stack:
- /home/user1/code/dragon/node_modules/prettier/index.js
- /home/user1/code/dragon/node_modules/@mixer/parallel-prettier/dist/index.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
    at resolve (node:internal/modules/helpers:187:19)
    at Object.transform (/home/user1/code/dragon/node_modules/prettier/index.js:18421:34)
    at run (/home/user1/code/dragon/node_modules/prettier/third-party.js:8418:53)
    at async cacheWrapper (/home/user1/code/dragon/node_modules/prettier/third-party.js:8294:22)
    at async Explorer.search (/home/user1/code/dragon/node_modules/prettier/third-party.js:8407:24)
    at async Promise.all (index 0) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/user1/code/dragon/node_modules/prettier/index.js',
    '/home/user1/code/dragon/node_modules/@mixer/parallel-prettier/dist/index.js'
  ]
}

Error: Cannot find module '@ajv-validator/config/prettierrc.json'
Require stack:
- /home/user1/code/dragon/node_modules/prettier/index.js
- /home/user1/code/dragon/node_modules/@mixer/parallel-prettier/dist/index.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
    at resolve (node:internal/modules/helpers:187:19)
    at Object.transform (/home/user1/code/dragon/node_modules/prettier/index.js:18421:34)
    at run (/home/user1/code/dragon/node_modules/prettier/third-party.js:8418:53)
    at async cacheWrapper (/home/user1/code/dragon/node_modules/prettier/third-party.js:8294:22)
    at async Explorer.search (/home/user1/code/dragon/node_modules/prettier/third-party.js:8407:24)
    at async Promise.all (index 0) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/user1/code/dragon/node_modules/prettier/index.js',
    '/home/user1/code/dragon/node_modules/@mixer/parallel-prettier/dist/index.js'
  ]
}
node_modules/collect-v8-coverage/index.js
node_modules/colord/colord.d.ts
node_modules/colord/index.js
...

EISDIR: illegal operation on a directory, read

Getting the error EISDIR: illegal operation on a directory, read

I just need Microsoft's input on this

So.... the picture I'm about to upload I know about the program I know how it operates but what I don't understand is that just a week or two ago this app was never showing up in my .txt file...... Is there a possibility someone else is viewing everything I'm doing on my Chromebook or is this something I shouldn't be concerned about because I'm constantly letting my computer unattended too while my wife's around not sure if she snuck one over on me... thanks

npm reports severity vulnerabilities in `glob-stream` version

When adding the latest version of parallel-prettier in the package-json of my project, npm tells me the following while doing npm audit:

# npm audit report

glob-parent  <5.1.2
Severity: moderate
Regular expression denial of service - https://npmjs.com/advisories/1751
No fix available
node_modules/glob-stream/node_modules/glob-parent
  glob-stream  >=5.3.0
  Depends on vulnerable versions of glob-parent
  node_modules/glob-stream
    @mixer/parallel-prettier  *
    Depends on vulnerable versions of glob-stream
    node_modules/@mixer/parallel-prettier

3 moderate severity vulnerabilities

Some issues need review, and may require choosing
a different dependency.