GithubHelp home page GithubHelp logo

vercel / pkg Goto Github PK

View Code? Open in Web Editor NEW
24.2K 277.0 994.0 40.35 MB

Package your Node.js project into an executable

Home Page: https://npmjs.com/pkg

License: MIT License

JavaScript 77.93% CSS 0.59% HTML 0.37% CoffeeScript 0.02% Pug 0.04% TypeScript 21.04% Batchfile 0.01%
nodejs binaries compiler cli

pkg's Introduction

pkg

Important

pkg has been deprecated with 5.8.1 as the last release. There are a number of successful forked versions of pkg already with various feature additions. Further, we’re excited about Node.js 21’s support for single executable applications. Thank you for the support and contributions over the years. The repository will remain open and archived.

This command line interface enables you to package your Node.js project into an executable that can be run even on devices without Node.js installed.

Use Cases

  • Make a commercial version of your application without sources
  • Make a demo/evaluation/trial version of your app without sources
  • Instantly make executables for other platforms (cross-compilation)
  • Make some kind of self-extracting archive or installer
  • No need to install Node.js and npm to run the packaged application
  • No need to download hundreds of files via npm install to deploy your application. Deploy it as a single file
  • Put your assets inside the executable to make it even more portable
  • Test your app against new Node.js version without installing it

Usage

npm install -g pkg

After installing it, run pkg --help without arguments to see list of options:

pkg [options] <input>

  Options:

    -h, --help           output usage information
    -v, --version        output pkg version
    -t, --targets        comma-separated list of targets (see examples)
    -c, --config         package.json or any json file with top-level config
    --options            bake v8 options into executable to run with them on
    -o, --output         output file name or template for several files
    --out-path           path to save output one or more executables
    -d, --debug          show more information during packaging process [off]
    -b, --build          don't download prebuilt base binaries, build them
    --public             speed up and disclose the sources of top-level project
    --public-packages    force specified packages to be considered public
    --no-bytecode        skip bytecode generation and include source files as plain js
    --no-native-build    skip native addons build
    --no-signature       skip signature of the final executable on macos
    --no-dict            comma-separated list of packages names to ignore dictionaries. Use --no-dict * to disable all dictionaries
    -C, --compress       [default=None] compression algorithm = Brotli or GZip

  Examples:

  – Makes executables for Linux, macOS and Windows
    $ pkg index.js
  – Takes package.json from cwd and follows 'bin' entry
    $ pkg .
  – Makes executable for particular target machine
    $ pkg -t node16-win-arm64 index.js
  – Makes executables for target machines of your choice
    $ pkg -t node16-linux,node18-linux,node16-win index.js
  – Bakes '--expose-gc' and '--max-heap-size=34' into executable
    $ pkg --options "expose-gc,max-heap-size=34" index.js
  – Consider packageA and packageB to be public
    $ pkg --public-packages "packageA,packageB" index.js
  – Consider all packages to be public
    $ pkg --public-packages "*" index.js
  – Bakes '--expose-gc' into executable
    $ pkg --options expose-gc index.js
  – reduce size of the data packed inside the executable with GZip
    $ pkg --compress GZip index.js

The entrypoint of your project is a mandatory CLI argument. It may be:

  • Path to entry file. Suppose it is /path/app.js, then packaged app will work the same way as node /path/app.js
  • Path to package.json. Pkg will follow bin property of the specified package.json and use it as entry file.
  • Path to directory. Pkg will look for package.json in the specified directory. See above.

Targets

pkg can generate executables for several target machines at a time. You can specify a comma-separated list of targets via --targets option. A canonical target consists of 3 elements, separated by dashes, for example node18-macos-x64 or node14-linux-arm64:

  • nodeRange (node8), node10, node12, node14, node16 or latest
  • platform alpine, linux, linuxstatic, win, macos, (freebsd)
  • arch x64, arm64, (armv6, armv7)

(element) is unsupported, but you may try to compile yourself.

You may omit any element (and specify just node14 for example). The omitted elements will be taken from current platform or system-wide Node.js installation (its version and arch). There is also an alias host, that means that all 3 elements are taken from current platform/Node.js. By default targets are linux,macos,win for current Node.js version and arch.

If you want to generate executable for different architectures, note that by default pkg has to run the executable of the target arch to generate bytecodes:

  • Linux: configure binfmt with QEMU.
  • macOS: possible to build x64 on arm64 with Rosetta 2 but not opposite.
  • Windows: possible to build x64 on arm64 with x64 emulation but not opposite.
  • or, disable bytecode generation with --no-bytecode --public-packages "*" --public.

macos-arm64 is experimental. Be careful about the mandatory code signing requirement. The final executable has to be signed (ad-hoc signature is sufficient) with codesign utility of macOS (or ldid utility on Linux). Otherwise, the executable will be killed by kernel and the end-user has no way to permit it to run at all. pkg tries to ad-hoc sign the final executable. If necessary, you can replace this signature with your own trusted Apple Developer ID.

To be able to generate executables for all supported architectures and platforms, run pkg on a Linux host with binfmt (QEMU emulation) configured and ldid installed.

Config

During packaging process pkg parses your sources, detects calls to require, traverses the dependencies of your project and includes them into executable. In most cases you don't need to specify anything manually.

However your code may have require(variable) calls (so called non-literal argument to require) or use non-javascript files (for example views, css, images etc).

require('./build/' + cmd + '.js');
path.join(__dirname, 'views/' + viewName);

Such cases are not handled by pkg. So you must specify the files - scripts and assets - manually in pkg property of your package.json file.

  "pkg": {
    "scripts": "build/**/*.js",
    "assets": "views/**/*",
    "targets": [ "node14-linux-arm64" ],
    "outputPath": "dist"
  }

The above example will include everything in assets/ and every .js file in build/, build only for node14-linux-arm64, and place the executable inside dist/.

You may also specify arrays of globs:

    "assets": [ "assets/**/*", "images/**/*" ]

Just be sure to call pkg package.json or pkg . to make use of package.json configuration.

Scripts

scripts is a glob or list of globs. Files specified as scripts will be compiled using v8::ScriptCompiler and placed into executable without sources. They must conform to the JS standards of those Node.js versions you target (see Targets), i.e. be already transpiled.

Assets

assets is a glob or list of globs. Files specified as assets will be packaged into executable as raw content without modifications. Javascript files may also be specified as assets. Their sources will not be stripped as it improves execution performance of the files and simplifies debugging.

See also Detecting assets in source code and Snapshot filesystem.

Options

Node.js application can be called with runtime options (belonging to Node.js or V8). To list them type node --help or node --v8-options.

You can "bake" these runtime options into packaged application. The app will always run with the options turned on. Just remove -- from option name.

You can specify multiple options by joining them in a single string, comma (,) separated:

pkg app.js --options expose-gc
pkg app.js --options max_old_space_size=4096
pkg app.js --options max-old-space-size=1024,tls-min-v1.0,expose-gc

Output

You may specify --output if you create only one executable or --out-path to place executables for multiple targets.

Debug

Pass --debug to pkg to get a log of packaging process. If you have issues with some particular file (seems not packaged into executable), it may be useful to look through the log.

Bytecode (reproducibility)

By default, your source code is precompiled to v8 bytecode before being written to the output file. To disable this feature, pass --no-bytecode to pkg.

Why would you want to do this?

If you need a reproducible build process where your executable hashes (e.g. md5, sha1, sha256, etc.) are the same value between builds. Because compiling bytecode is not deterministic (see here or here) it results in executables with differing hashed values. Disabling bytecode compilation allows a given input to always have the same output.

Why would you NOT want to do this?

While compiling to bytecode does not make your source code 100% secure, it does add a small layer of security/privacy/obscurity to your source code. Turning off bytecode compilation causes the raw source code to be written directly to the executable file. If you're on *nix machine and would like an example, run pkg with the --no-bytecode flag, and use the GNU strings tool on the output. You then should be able to grep your source code.

Other considerations

Specifying --no-bytecode will fail if there are any packages in your project that aren't explicitly marked as public by the license in their package.json. By default, pkg will check the license of each package and make sure that stuff that isn't meant for the public will only be included as bytecode.

If you do require building pkg binaries for other architectures and/or depend on a package with a broken license in its package.json, you can override this behaviour by either explicitly whitelisting packages to be public using --public-packages "packageA,packageB" or setting all packages to public using --public-packages "*"

Build

pkg has so called "base binaries" - they are actually same node executables but with some patches applied. They are used as a base for every executable pkg creates. pkg downloads precompiled base binaries before packaging your application. If you prefer to compile base binaries from source instead of downloading them, you may pass --build option to pkg. First ensure your computer meets the requirements to compile original Node.js: BUILDING.md

See pkg-fetch for more info.

Compression

Pass --compress Brotli or --compress GZip to pkg to compress further the content of the files store in the exectable.

This option can reduce the size of the embedded file system by up to 60%.

The startup time of the application might be reduced slightly.

-C can be used as a shortcut for --compress .

Environment

Var Description
PKG_CACHE_PATH Used to specify a custom path for node binaries cache folder. Default is ~/.pkg-cache
PKG_IGNORE_TAG Allows to ignore additional folder created on PKG_CACHE_PATH matching pkg-fetch version
MAKE_JOB_COUNT Allow configuring number of processes used for compiling

Examples

# 1 - Using export
export PKG_CACHE_PATH=/my/cache
pkg app.js

# 2 - Passing it before the script
PKG_CACHE_PATH=/my/cache pkg app.js

Usage of packaged app

Command line call to packaged app ./app a b is equivalent to node app.js a b

Snapshot filesystem

During packaging process pkg collects project files and places them into executable. It is called a snapshot. At run time the packaged application has access to snapshot filesystem where all that files reside.

Packaged files have /snapshot/ prefix in their paths (or C:\snapshot\ in Windows). If you used pkg /path/app.js command line, then __filename value will be likely /snapshot/path/app.js at run time. __dirname will be /snapshot/path as well. Here is the comparison table of path-related values:

value with node packaged comments
__filename /project/app.js /snapshot/project/app.js
__dirname /project /snapshot/project
process.cwd() /project /deploy suppose the app is called ...
process.execPath /usr/bin/nodejs /deploy/app-x64 app-x64 and run in /deploy
process.argv[0] /usr/bin/nodejs /deploy/app-x64
process.argv[1] /project/app.js /snapshot/project/app.js
process.pkg.entrypoint undefined /snapshot/project/app.js
process.pkg.defaultEntrypoint undefined /snapshot/project/app.js
require.main.filename /project/app.js /snapshot/project/app.js

Hence, in order to make use of a file collected at packaging time (require a javascript file or serve an asset) you should take __filename, __dirname, process.pkg.defaultEntrypoint or require.main.filename as a base for your path calculations. For javascript files you can just require or require.resolve because they use current __dirname by default. For assets use path.join(__dirname, '../path/to/asset'). Learn more about path.join in Detecting assets in source code.

On the other hand, in order to access real file system at run time (pick up a user's external javascript plugin, json configuration or even get a list of user's directory) you should take process.cwd() or path.dirname(process.execPath).

Detecting assets in source code

When pkg encounters path.join(__dirname, '../path/to/asset'), it automatically packages the file specified as an asset. See Assets. Pay attention that path.join must have two arguments and the last one must be a string literal.

This way you may even avoid creating pkg config for your project.

Native addons

Native addons (.node files) use is supported. When pkg encounters a .node file in a require call, it will package this like an asset. In some cases (like with the bindings package), the module path is generated dynamicaly and pkg won't be able to detect it. In this case, you should add the .node file directly in the assets field in package.json.

The way Node.js requires native addon is different from a classic JS file. It needs to have a file on disk to load it, but pkg only generates one file. To circumvent this, pkg will create a temporary file on the disk. These files will stay on the disk after the process has exited and will be used again on the next process launch.

When a package, that contains a native module, is being installed, the native module is compiled against current system-wide Node.js version. Then, when you compile your project with pkg, pay attention to --target option. You should specify the same Node.js version as your system-wide Node.js to make compiled executable compatible with .node files.

Note that fully static Node binaries are not capable of loading native bindings, so you may not use Node bindings with linuxstatic.

API

const { exec } = require('pkg')

exec(args) takes an array of command line arguments and returns a promise. For example:

await exec(['app.js', '--target', 'host', '--output', 'app.exe']);
// do something with app.exe, run, test, upload, deploy, etc

Troubleshooting

Error: ENOENT: no such file or directory, uv_chdir

This error can be caused by deleting the directory the application is run from. Or, generally, deleting process.cwd() directory when the application is running.

Error: ERR_INSPECTOR_NOT_AVAILABLE

This error can be caused by using NODE_OPTIONS variable to force to run node with the debug mode enabled. Debugging options are disallowed , as pkg executables are usually used for production environments. If you do need to use inspector, you can build a debuggable Node.js yourself.

Error: require(...).internalModuleStat is not a function

This error can be caused by using NODE_OPTIONS variable with some bootstrap or node options causing conflicts with pkg. Some IDEs, such as VS Code, may add this env variable automatically.

You could check on Unix systems (Linux/macOS) in bash:

$ printenv | grep NODE

Advanced

exploring virtual file system embedded in debug mode

When you are using the --debug flag when building your executable, pkg add the ability to display the content of the virtual file system and the symlink table on the console, when the application starts, providing that the environement variable DEBUG_PKG is set. This feature can be useful to inspect if symlinks are correctly handled, and check that all the required files for your application are properly incorporated to the final executable.

$ pkg --debug app.js -o output
$ DEBUG_PKG=1 output

or

C:\> pkg --debug app.js -o output.exe
C:\> set DEBUG_PKG=1
C:\> output.exe

Note: make sure not to use --debug flag in production.

pkg's People

Contributors

alexanderbartels avatar alexk111 avatar blackyuzia avatar brianunlam avatar charliea21 avatar commanderroot avatar danetheory avatar dependabot[bot] avatar eltociear avatar erossignon avatar evilrabbit avatar greenkeeperio-bot avatar hipstersmoothie avatar ignatiusmb avatar igorklopov avatar jesec avatar kldzj avatar leerob avatar leo avatar martinstarman avatar mika-fischer avatar mikescops avatar mojoaxel avatar onip avatar paul-marechal avatar phated avatar rauchg avatar robertslando avatar symbitic avatar zvin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pkg's Issues

Pkg: 'callsites' package is temporarily unsupported

I packaged my project, and ran into an issue that inhibits me from successfully running the output binary. One package I use heavily relies on parent-module, which in turn uses callsites, which is what causes the error. Honestly, I was going to be very impressed if it was able to deal with the parent-module module :). Here's the error I get when running the binary.

$ pkg --targets node6-macos-x64 .
... Packages, with some "Cannot resolve" warnings which I've added to `scripts` ...

$ ./my-project-macos
/snapshot/my-project/node_modules/callsites/index.js:1
(function (exports, require, module, __filename, __dirname) { module.exports = function() {throw new Error("Pkg: 'callsites' package is temporarily unsupported. Please create a github issue.")};
                                                                                                 ^

Error: Pkg: 'callsites' package is temporarily unsupported. Please create a github issue.
    at module.exports (/snapshot/my-project/node_modules/callsites/index.js:1:98)
    at module.exports (/snapshot/my-project/node_modules/parent-module/index.js:5:15)
    at Object.exports.using (/snapshot/my-project/node_modules/haute-couture/lib/index.js:19:39)
    at Object.console./snapshot/my-project/lib/index.js.dev (evalmachine.<anonymous>:0)
    at Module._compile (evalmachine.<anonymous>:0)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.require (module.js:497:17)

It instructed me to make a github issue, so here I am! Let me know if I can be useful to help solve the issue, or if I can provide any more information. Thanks!

Project license

I don't see any license attached to this repository; could you please add an open source license (like the MIT license) so we can be sure we can use this legally?

Thanks in advance.

Question

Hi sorry for asking this, I just want to double check before I get all excited.
So I have a node 6.x app running in production and my app was the sole reason my custom had to install node in the first place.

Could I uninstall node 6 on the server, develop the next iteration in say node 7 and pkg it to an executable express server? Would that actually work?

Error triggered on some files (.md, .npmignore, LICENCE)

Trying to pack a project which includes jade (pug), these errors are triggered:

Error! Unexpected token (16:0)
../node_modules/jade/.npmignore
Error! Unexpected token (1:4)
../node_modules/jade/History.md
Error! Unexpected token (1:5)
../node_modules/jade/LICENSE

Build cmd: pkg package.json --options expose-gc --targets node7-linux-armv7 --out-dir build

jade dependency in package.json: "jade": "^1.11.0",

out of memory

Trying to pack this project https://github.com/jbdemonte/Retro-Manager with this commande:

pkg package.json --options expose-gc --targets node7-linux-armv7 --out-dir build

I'm getting this error:

FATAL ERROR: Malloced operator new Allocation failed - process out of memory

Error! Error: /home/pi/.pkg-cache/v1.9/fetched-v7.6.0-linux-armv7 failed with code null
at ChildProcess. (/usr/lib/node_modules/pkg/lib-es5/producer.js:39:25)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:194:7)
at maybeClose (internal/child_process.js:899:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)

How to access filesystem's /snapshot directory? (not bundled assets)

If I use pkg to make a CLI tool that manipulates files, then it's possible that users might try to use it on a /snapshot directory on their filesystem. (rare but possible, and for a CLI tool, I'd need to cover all cases) How do I read and write to the host filesystem's /snapshot directory with a pkg-bundled executable? I couldn't find this in the documentation, but maybe I missed the section that explains this.

Example:

cd /snapshot
vim index.js
my-pkg-bundled-utility uploadFileToWebsite index.js

Compilation error with buffer.

Error happend in compilation of simple index.js with few functions.
`C:\Users\Kradre\Desktop\mqttBroker>pkg index.js

Targets not specified. Assuming:
node4-linux-x64, node4-macos-x64, node4-win-x64
C:\Users\Kradre\AppData\Roaming\npm\node_modules\pkg\lib-es5\producer.js:89
return Buffer.alloc(padding);
^
TypeError: Buffer.alloc is not a function
at paddingBuffer (C:\Users\Kradre\AppData\Roaming\npm\node_modules\pkg\lib-es5\producer.js:89:17)
at C:\Users\Kradre\AppData\Roaming\npm\node_modules\pkg\lib-es5\producer.js:25:48
at MultiStream._next (C:\Users\Kradre\AppData\Roaming\npm\node_modules\pkg\node_modules\multistream\index.js:81:27)
at ReadStream.onEnd (C:\Users\Kradre\AppData\Roaming\npm\node_modules\pkg\node_modules\multistream\index.js:119:10)
at ReadStream.g (events.js:260:16)
at emitNone (events.js:72:20)
at ReadStream.emit (events.js:166:7)
at endReadableNT (_stream_readable.js:913:12)
at nextTickCallbackWith2Args (node.js:442:9)
at process._tickCallback (node.js:356:17)`

I think...

I would like to try, uh ... I still do not use github, thanks.

The animate.css module: Error! Unexpected character '@'

I used the express example of the pkg project to test the animate.css module I used in my project, installed by: npm i animate.css -S, and when I ran the command: pkg package.json, it came out the result:

> Error! Unexpected character '@' (1:0)
  /home/test/pkg-master/examples/express/node_modules/animate.css/animate.css

Maybe it isn't the place to ask question, but I tried according to the readme of pkg, and with no luck. It didn't work, whether I modified the package.json by adding the assets part or other way, Does anybody know how to solve this, tell me please, thx.

Use `start` script

The canonical default executable is start script in the package.json file, that by default is server.js file, not bin entry since you can define several binaries in a single package.

Mention drawbacks in the readme

As I understand this is a rebranding of the enclose module? Would love to know how this happened btw.

But, in original documentation, here – http://enclosejs.com/#compilation-srsly it says that resulting binary will be at least 2x slower, is that still the case? And if so, can you mention that in the readme? It would be better I think to also document how this sort of measurements was performed.

As well as any other possible drawbacks, besides performance and size

Error: File or directory. Please recompile adding it as asset or script.

A quick support - I am doing a test compile. Here is my package .json script I have added.

"pkg": {
    "scripts": "build/**/*.js",
    "assets": "views/**/*"
  }

Here is the folder structure

folderstr

When I create a .exe it does create a exe but it gives an error saying this:

Error: File or directory 'C:\SGems\fullstack\5\views\index.html' was not included into executable at compilation stage. Please recompile adding it as asset or script.
at error_ENOENT (evalmachine.:0:0)
at findNativeAddonForStat (evalmachine.:0:0)
at statFromSnapshot (evalmachine.:0:0)
at Object.fs.stat (evalmachine.:0:0)
at SendStream.sendFile (C:\snapshot\SGems\fullstack\5\node_modules\send\index.js:729:6)
at SendStream.pipe (C:\snapshot\SGems\fullstack\5\node_modules\send\index.js:603:8)
at sendfile (C:\snapshot\SGems\fullstack\5\node_modules\express\lib\response.js:1057:8)
at ServerResponse.sendFile (C:\snapshot\SGems\fullstack\5\node_modules\express\lib\response.js:418:3)
at evalmachine.:0:0
at Layer.handle [as handle_request] (C:\snapshot\SGems\fullstack\5\node_modules\express\lib\router\layer.js:95:5)

Also, is there a way to keep the server running in quiet mode? Any options?

Have pkg ignore files or directories (.pkgignore?)

Hello!

I was wondering if there was a way to ignore a specific missing file.
Summary:
pkg tries to resolve all path.join calls and we can be punished for a missing file that might not be important. A .pkgignore file might make sense for this?

Long version:
Here's my use case because maybe there's another/cleaner way of fixing this.
When I try to build my binaries I get this error:

> Error! Cannot stat, ENOENT
  /Users/zmarouf/my_module/node_modules/tabtab/.completions/cache.json
  The file was required from '/Users/zmarouf/my_module/node_modules/tabtab/src/cache.js'

This is the problematic code:

var cachefile = path.join(__dirname, '../.completions/cache.json');
...
this._cache = exists(cachefile) ? require(cachefile) : { timestamp: Date.now(), cache: {} };

My hacky way of fixing this right now is to just touch the missing file but that's—well— ugly and not very practical.

What do you think? What are your suggestions on this? :)
I did a quick search in the issues for a similar problem but it didn't yield anything.
If there's already an open/closed discussion about this then could you please direct me to it?

Thank you in advance for your time!

Difference from nexe?

Could you highlight in a nutshell how pkg is different?
https://github.com/nexe/nexe

so far i'm seeing Nexe rebuilding node with source, and pkg inlining the source into a new binary (thus not requiring a node rebuild)

Thanks!

fs.readFile with encoding option can't access files in /snapshot

Hi, all.

I think that zeit's pkg is cool solution to everyone who want to simplify their nodejs resources to one executable.
Thanks for efforts of pkg maintainers in zeit :)

I found fs.readFile with { encoding: 'utf8' } as options can't read anything in /snapshot, but
but in fs.readFile with only "utf8" string as options, there is not such a problem.
It seems that in executable, fs.readFile does not handle encoding attribute in option of object type.

Interestingly, fs.readFileSync works well in both case.

The following is test code.

const path = require('path')
const fs = require('fs')

const meta = path.join(__dirname, './meta.txt')

let syncdata = fs.readFileSync(meta, { encoding: 'utf8' })
console.log('readFileSync case with { encoding: \'utf8\'} option: ', syncdata)

syncdata = fs.readFileSync(meta, 'utf8')
console.log('readFileSync case with \'utf8\' option: ', syncdata)

fs.readFile(meta, { encoding: 'utf8' }, (err, data) => {
  if (err) {
    console.log(err)
    return 
  }
  console.log('readFile case with { encoding: \'utf8\'} option: ', data)
})

fs.readFile(meta, 'utf8', (err, data) => {
  if (err) {
    console.log(err)
    return 
  }
  console.log('readFile case with \'utf8\' option: ', data)
})

the result is the following.

readFileSync case with { encoding: 'utf8'} option: # this is test text
readFileSync case with 'utf8' option: # this is test text
readFile case with 'utf8' option: # this is test text
{ Error: ENOENT: no such file or directory, open '/snapshot/home/nemoux/nemosnow/snow-apps/pkgsnapshot-test/meta.txt'
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/snapshot/home/nemoux/nemosnow/snow-apps/pkgsnapshot-test/meta.txt' }

I executed the following command after extracting the following.

$ pkg index.js -t latest-linux-x64

pkgsnapshot-test.zip

Is it a bug or right behavior?

Release with github releases

It would be awesome to set up github releases for the project in order to work well with other package managers (like homebrew and such).

Possibility to reduce packages size by 4x

Hi! Just done a quick test on windows builds. Bundle size can be reduced from 24,964 KB to 6,646 KB (3.75x smaller) with doing simple xz compression. This may even help more when having more npm dependencies. Adding this option may be super useful for small and embedded CLI/Webservers :) I've just opened this discussion before going further to know others idea on this.

UPDATE: To be clear, this topic and what i am talking about is SFX packages not js minification & optimizing.

Unable to package CLI app that uses "image-size"

TLDR: I found Enclose, tried it and got an error. Found an issue report over there suggesting to use this instead. But I get the exact same message with Pkg.

Note: For the record, I do actually want to replace image-size with something that lets me do true image comparisons (currently I'm using the MD5 file hash, which is getting messy).

My console output is as follows:

D:\dev-sandbox\spotlight-extractor>pkg -t node6-win index.js
> Fetching base Node.js binaries to: ~/.pkg-cache
  fetched-v6.10.0-win-x86      [====================] 100%
> Warning Cannot resolve ''./types/' + type'
  D:\dev-sandbox\spotlight-extractor\node_modules\image-size\lib\index.js
  Use a string literal as argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.
> Warning Cannot resolve ''./types/' + type'
  D:\dev-sandbox\spotlight-extractor\node_modules\image-size\lib\detector.js
  Use a string literal as argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.

I have a few other projects and ideas that I want to package for distribution, but they're not yet ready for that just yet.. So for the sake of understanding this issue I'll ask my question using this project as reference for now.

Obviously modifying the library that causes the problem is one option, but is that the best way to handle this situation? Is there anything else that can be done to work around this?

Mention 'cross-cpu-compilation' limitation in docs

Hi, I am cross-compiling for arm6 or arm7 on 64-bit Ubuntu host.

Regardless of if I use armv6 or armv7, and regardless of if I use node6 or node7, I always get the following error:

/home/msolters/.pkg-cache/v1.9/fetched-v6.10.0-linux-armv7: 1: /home/msolters/.pkg-cache/v1.9/fetched-v6.10.0-linux-armv7: Syntax error: word unexpected (expecting ")")
> Error! Error: Not able to compile for 'armv7' here
    at Socket.<anonymous> (/home/msolters/.nvm/versions/node/v6.10.0/lib/node_modules/pkg/lib-es5/producer.js:43:25)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at onwriteError (_stream_writable.js:346:10)
    at onwrite (_stream_writable.js:364:5)
    at WritableState.onwrite (_stream_writable.js:90:5)
    at fireErrorCallbacks (net.js:468:13)
    at Socket._destroy (net.js:509:3)
    at WriteWrap.afterWrite (net.js:803:10)

Incorrect project description

Apologies if this is the wrong place to post this, but I couldn't find a more relevant repo.

I was browsing your site and came across this:

pkg

This reads "Singe-Command compiler" instead of the newer description "Package your Node.js project into an executable".

I cloned https://github.com/zeit/github-projects and verified that the latest description is coming through. Just wanted to let you know you may want to refresh your cache, if this is still powering your OSS page.

'pkg' binary name conflict

Hi!

I'm currently looking at various node packagers, and it's very nice to see that yours supports FreeBSD (unlike nar which only supports building executables for Linux, Darwin and Solaris — the ones with node binaries published by the node project itself).

However… FreeBSD's native package manager is literally called pkg ;)

Code sign fails

I cannot package this binary into my app, because the binary cannot be codesigned by XCode.
This works on other binaries and files, so there must be an issue specifically around the binaries that pkg creates.

To reproduce, create a simple index.js with console.log('Hello world').

pkg -t node7-darwin index.js -o test

The resulting binary cannot be code signed.

$ codesign -vvvv --force --sign '<apple developer identity>' ./test
./test: main executable failed strict validation

What is the relationship of pkg to enclose?

Does pkg supersede enclose? There appears to be a relationship between the two projects, but I see that the github repo for enclose hasn't been updated in a while. Also pkg appears to be fully open source, whereas enclose is (partially?) proprietary.

Would be useful to have an explainer on this somewhere to reduce confusion.

Also, wow this project is cool. 😄

vue-quill-editor module: Error! Unexpected token (1:0)

Testing with the vue wrapper of quill editor, while the vue-quill-editor contains a vue component in the require expression, so the pkg run out with the result below:

> Error! Unexpected token (1:0)
  /home/test/jzver/node_modules/vue-quill-editor/src/editor.vue

and the index.js file of the vue-quill-editor just like:

/**
 * Vue-Quill-Editor
 * @author Surmon.me
 */

window.Quill = require('quill/dist/quill.js')
var quillEditor = require('./editor.vue')
var VueQuillEditor = {
  Quill: Quill,
  quillEditor: quillEditor,
  install: function(Vue) {
    Vue.component('quill-editor', quillEditor)
  }
}

module.exports = VueQuillEditor

Does it just add the editor.vue file to assets of package.json could solve this problem? I tried but failed, maybe I was wrong about the way to use the assets in package.json file. Any suggestion would be appreciated, thx a lot.

Class.toString() returns incorrect value

When I have a plain JS class and try to turn it into a string, my program hangs. I'm on a Mac, on node v6. I think the demo below will provide enough information to get started.

Contents of test.js

'use strict';

const arrow = () => null;
const Class = class {};

console.log('arrow');
console.log(typeof arrow);
console.log(arrow);
console.log(arrow.toString());

console.log('Class');
console.log(typeof Class);
console.log(Class);
console.log(Class.toString());

console.log('end');

Demo

$ pkg --targets node6-macos-x64 test.js

$ ./test 
arrow
function
[Function: arrow]
function arrow() { [native code] }
Class
function
[Function: Class]
^C # Program hung here!

$ node -v
v6.4.0

$ node test.js
arrow
function
[Function]
() => null
Class
function
[Function]
class {}
end

pkg attempts to build unnecessary and illogical targets

Upon specifying -t node7-linux-armv7 as the only target on a MacOS host, pkg downloads the armv7 base binary, but then starts to (attempt to) build a macos-armv7 base binary from source. This is illogical and unnecessary. I stopped it after noticing what it was doing, so I don't know if it would subsequently attempt to make a win-armv7 binary as well.
Transcript of attempted build:

$ pkg . --targets node7-linux-armv7
> Fetching base Node.js binaries to: ~/.pkg-cache
  fetched-v7.6.0-linux-armv7   [====================] 100%
  fetched-v7.6.0-macos-armv7   [====================] 100%
> Error! 404 Not Found
  https://github.com/zeit/pkg-fetch/releases/download/v1.9/uploaded-v1.9-node-v7.6.0-macos-armv7
> Asset 'v1.9/uploaded-v1.9-node-v7.6.0-macos-armv7' not found by direct link
> Not found in GitHub releases:
  {"tag":"v1.9","name":"uploaded-v1.9-node-v7.6.0-macos-armv7"}
> Building base binary from source:
  built-v7.6.0-macos-armv7
> Cloning Node.js repository from GitHub...
  git                          [========            ] 40%^C

Update: it does not attempt to make a macos-armv7 target when running on an actual linux-armv7 host (i.e., a Raspberry Pi 3). So it's not using the default linux,macos,win targets, but apparently trying to build a host-native version as well as the specified target.
Update 2: Upon attempting a node6-macos-x64 build on the armv7 host, the same pattern followed where pkg attempted to build a host-native version of the target arch (in this case, node6-linux-x64) but crashed in the attempt:

$ pkg . -t node6-macos-x64
> Fetching base Node.js binaries to: ~/.pkg-cache
  fetched-v6.10.0-macos-x64    [====================] 100%
  fetched-v6.10.0-linux-x64    [====================] 100%
/home/pi/.pkg-cache/v1.9/fetched-v6.10.0-linux-x64: 1: /home/pi/.pkg-cache/v1.9/fetched-v6.10.0-linux-x64: cannot create $�x@��x@@%@@@@@�@@@@L�kL�k �k����(s� 0�k0��0��0@@DDP�td�O住住����Q�td/lib64/ld-linux-x86-64.so.2GNUGNU�N�ޞ:^�m�zw���3�m���A�3G
       : Directory nonexistent
/home/pi/.pkg-cache/v1.9/fetched-v6.10.0-linux-x64: 1: /home/pi/.pkg-cache/v1.9/fetched-v6.10.0-linux-x64: �ELF: not found
/home/pi/.pkg-cache/v1.9/fetched-v6.10.0-linux-x64: 4: /home/pi/.pkg-cache/v1.9/fetched-v6.10.0-linux-x64: Syntax error: ")" unexpected
> Error! Error: Not able to compile for 'x64' here
    at Socket.<anonymous> (/home/pi/.npm/lib/node_modules/pkg/lib-es5/producer.js:43:25)
    at emitOne (events.js:90:13)
    at Socket.emit (events.js:182:7)
    at onwriteError (_stream_writable.js:313:10)
    at onwrite (_stream_writable.js:331:5)
    at WritableState.onwrite (_stream_writable.js:89:5)
    at fireErrorCallbacks (net.js:448:13)
    at Socket._destroy (net.js:486:3)
    at WriteWrap.afterWrite (net.js:769:10)

Error when using googleapis

My app uses the Google Sheets API.

pkg generates the binaries ok but throws the following error when I run that binary:


/snapshot/Users/conoroneill/gitwork/yunmai-data-extract/node_modules/googleapis/lib/googleapis.js:62
fs.readdirSync(path.join(__dirname, '../apis')).forEach(function (file) {
   ^

Error: Directory '/Users/conoroneill/gitwork/yunmai-data-extract/node_modules/googleapis/apis' was not included into executable at compilation stage. Please recompile adding it as asset or script.
    at error_ENOENT (evalmachine.<anonymous>:0)
    at readdirFromSnapshot (evalmachine.<anonymous>:0)
    at Object.fs.readdirSync (evalmachine.<anonymous>:0)
    at Object.<anonymous> (/snapshot/Users/conoroneill/gitwork/yunmai-data-extract/node_modules/googleapis/lib/googleapis.js:62:4)
    at Module._compile (module.js:570:32)
    at Module._compile (evalmachine.<anonymous>:0)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)

I've tried many variations on

 "pkg": {
    "assets": "node_modules/googleapis/apis/**/*"
  }

But when I use --debug, I don't see the files being added. Am I missing something obvious in the path or glob?

Does pkg compiling prevent source code from being extractable?

In the README you mention that scripts that are added as scripts in the config are compiled using the V8 compiler. Is that also true for regular dependencies that are found from the entry point?

And does this compilation prevent source code from being accessible in the package?

My use case here is, that I have a private key string in my source code, but when I ran strings on the resulting executable, the string was still there in plain text.

Thanks!

--loglevel param

Old enclose had --loglevel param where you could have disabled warning level log messages for example. I can't seem to find this in the new version. Was it removed for some reason?

In some places I am not using string literals with require calls, but including those files using scripts config option. Hence those warnings aren't really interesting to me and I am well aware of the functionality.

Could this be combined with Electron?

Practically, the only time you want to distribute a commercial app and use some sort of code protection is with desktop apps. So it would make a lot of sense to combine this with Electron. Are there any plans for that?

pkg can't work with zeromq module and grpc module

I use zeromq and grpc module in my project, while I tried to use pkg to bundle my project, it gave the warning like below:

> Warning Cannot resolve 'binding_path'
  /home//test/jzver/node_modules/grpc/src/node/src/grpc_extension.js
  Use a string literal as argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.
> Warning Cannot include native addon into executable.
  /hometest/jzver/node_modules/zeromq/build/Release/zmq.node
  The addon file must be distributed with executable.

The grpc_extension.js looks like:

var binary = require('node-pre-gyp/lib/pre-binding');
var path = require('path');
var binding_path =
    binary.find(path.resolve(path.join(__dirname, '../../../package.json')));
var binding = require(binding_path);

module.exports = binding;

While the grpc module should be compile to node C++ addon with the grpc_extension.js, and to the zeromq module, when the compile job is done, it produce the zmq.node to be used in node project. I don't know how to handle node modules like these by pkg.
And my node version is: 6.10.0
I used the command like this: pkg -t node6-linux ./server/index.js
Thanks for amazing work with pkg

Error! connect ETIMEDOUT 54.231.33.195:443

[root@iZ94hmrh5tzZ ra-broker]# pkg index.js

Targets not specified. Assuming:
node7-linux-x64, node7-macos-x64, node7-win-x64
Fetching base Node.js binaries to: ~/.pkg-cache
fetched-v7.6.0-linux-x64 [====================] 100%
Error! connect ETIMEDOUT 54.231.33.195:443
Asset 'v1.9/uploaded-v1.9-node-v7.6.0-linux-x64' not found by direct link
fetched-v7.6.0-linux-x64 [====================] 100%
Error! connect ETIMEDOUT 54.231.33.195:443

pkg errors with "No available node version satisfies 'node5'"

Running pkg . with a package.json containing no target (i.e. expecting default/current version):

  ,"pkg":{"scripts":"engine/**/*.js","assets":"engine/**/*"}
  ,"bin":"engine/app.js"

Returns Error! No available node version satisfies 'node5'

Trying with different targets specified such as "node5.0" or "node5-win" or "v5.0.0" or "latest-v5.x" or returns the following:

Error! No available node version satisfies 'node5.0'
Error! No available node version satisfies 'node5'
Error! Unknown token 'v5.0.0' in 'v5.0.0'
Error! Unknown token 'v5.x' in 'latest-v5.x'

I'm pretty sure there was a version of Node 5 downloaded when installing pkg, but I can't find where it was placed and there doesn't appear to be a command to list available Node versions / valid targets?

Performance comparison of generated binary vs running through node

I suppose running the binary instead of node with js files will be slow as v8 makes many optimizations which may not be possible at compile time. Are their any statistics of how slow this compiled code runs in comparison to the default interpretted (JIT) way running node apps

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.