Support for aix ppc64 BE about esbuild

[evanw]

I just published a version of esbuild that works on aix ppc64 BE in theory. Please try it out in that environment and let me know if it works or not. Something like npm i [email protected] && npx esbuild --version should be working now.

[abmusse]

@evanw
Just tested this on AIX 7.2 system and confirm install is successful!

$ uname -vrs
AIX 2 7

$ npm i [email protected]

added 2 packages, and audited 3 packages in 7s

found 0 vulnerabilities

$ npx esbuild --version
0.19.10

[abmusse]

@bskimball

Yes AIX complied go programs won't work properly on IBM i due to many issues including the mmap one you mentioned.

For now the best solution would be to run esbuild to bundle your code on another machine then serve the files on your IBM i.

[merceyz]

If Node.js runs on the machine you could always try esbuild-wasm.

[merceyz]

Yes, see

esbuild/Makefile

Lines 55 to 62 in 40711af

	# This uses "env -i" to run in a clean environment with no environment
	# variables. It then adds some environment variables back as needed.
	# This is a hack to avoid a problem with the WebAssembly support in Go
	# 1.17.2, which will crash when run in an environment with over 4096
	# bytes of environment variable data such as GitHub Actions.
	test-wasm-node: esbuild
	env -i $(shell go env) PATH="$(shell go env GOROOT)/misc/wasm:$(PATH)" GOOS=js GOARCH=wasm go test ./internal/...
	node scripts/wasm-tests.js

.

[evanw]

Do you have access to such a system and can conform that esbuild can build and run on that system (e.g. installing go and running make test-go in this repo passes all tests)? I don't have access to such a system myself.

[merceyz]

I don't but https://github.com/nodejs/citgm does, maybe they'd be willing to test it, if not a WebAssembly fallback package should presumably work and resolve the issue in the comment I linked to.

[richardlau]

Do you have access to such a system and can conform that esbuild can build and run on that system (e.g. installing go and running make test-go in this repo passes all tests)? I don't have access to such a system myself.

I've asked @abmusse from IBM to take a look. We think it should be okay, based on https://go.dev/doc/install/source#environment listing $GOOS=AIX and $GOARCH=ppc64 as a valid combination but @abmusse will try to verify the tests pass on an AIX system.

[bskimball]

Can you do the same for os400? It runs PASE so if esbuild runs on AIX it will also run in the pase environment. I do have access to the server
$GOOS=os400 and $GOARCH=ppc64

[evanw]

This issue came about from someone trying to install esbuild using node/npm on aix, which is possible because aix is one of the operating systems that node supports: https://nodejs.org/api/process.html#processplatform. However, os400 is not on this list, which might mean that running an official build of node on os400 is not possible (although there still might still be an unofficial build of node/npm for os400).

If you just want to run the esbuild binary executable on os400 (or any other platform that the Go compiler toolchain supports) without using node, then you should already be able to compile esbuild for yourself for that platform. You don't need any changes to esbuild to do that.

If you want to use esbuild with node via the package that's published to npm, then it needs to be possible to run npm on that platform (so you can run the installer to download that prebuilt version of esbuild). If this is possible, then I'm assuming that means there's an unofficial build of node/npm for os400 since os400 is not one of node's officially-supported operating systems. In that case I'd need to know what values that unofficial build uses to identify the platform (specifically what node -p '[process.platform, os.arch(), os.endianness()]' says, which is how esbuild's installer identifies the current platform). This is what the error message in the original post provided, which is what made it possible for me to know how to add support for this particular platform without being able to use it myself.

[bskimball]

os400 more commonly rreferred to as IBMi has an @ibm-release maintained version of Node that runs in the shell environment and is installed via yum (https://ibmi-oss-docs.readthedocs.io/en/latest/nodejs/README.html) .

node -p '[process.platform, os.arch(), os.endianness()]' results in the following -> ['os400','ppc64','BE']

[evanw]

Thanks. That should be enough info for me to proceed.

[evanw]

Wait actually Go doesn't appear to support building for os400 even on the latest release:

$ go version
go version go1.21.6 linux/amd64

$ GOOS=os400 GOARCH=ppc64 go build ./cmd/esbuild
go: unsupported GOOS/GOARCH pair os400/ppc64

$ go tool dist list | grep os400
(no output)

So I'm not able to build esbuild for this OS because Go doesn't support it.

$GOOS=os400 and $GOARCH=ppc64

How are you able to build esbuild using these parameters? Do you also have a custom Go compiler?

[bskimball]

I haven't tried any go programs on there personally. I have read that go compiled for aix will run in the PASE environment, but I've never tried it.

This particular issue was from me trying to run vite ssr on IBMi. install.js throws the unsupported platform error.

This may fix my issue by adding to nodePlatform.ts

export const knownUnixlikePackages: Record<string, string> = {
   'aix ppc64 BE': '@esbuild/aix-ppc64',
+  'os400 ppc64 BE': '@esbuild/aix-ppc64',   
   'android arm64 LE': '@esbuild/android-arm64',

PASE on IBMi is a unix shell similar to aix. If I get time tomorrow I may try and fork to test.

[evanw]

To save you some forking work: I believe the only thing you would need to confirm is whether the binary executable in https://registry.npmjs.org/@esbuild/aix-ppc64/-/android-arm-0.19.11.tgz works correctly on os400 or not. You probably don't need to fork esbuild. I assume that node will run correctly on os400 so you shouldn't need to test that, but I need to know whether the executable will work or not. It may not work correctly (e.g. this issue is open).

[bskimball]

It does not run unfortunately. Due to the mmap pointers. Doesn't look like IBM has any plans on supporting Go either. Thanks for your help with this issue.

[bskimball]

@bskimball

Yes AIX complied go programs won't work properly on IBM i due to many issues including the mmap one you mentioned.

For now the best solution would be to run esbuild to bundle your code on another machine then serve the files on your IBM i.

That's what I do now. I was hoping to eventually run ssr on IBMi with esbuild

[bskimball]

If Node.js runs on the machine you could always try esbuild-wasm.

I might try that, thanks.

[abmusse]

If Node.js runs on the machine you could always try esbuild-wasm.

Yes Node.js is supported on IBM i.

I think giving esbuild-wasm a shot is worth it, will try to test it on IBM i.

[bskimball]

Looks like the wasm packages do run on IBMi. I'm using vite so my package.json lists the following overrides for esbuild and rollup.

  "overrides": {
    "esbuild": "npm:esbuild-wasm@latest",
    "rollup": "npm:@rollup/wasm-node"
  },

** update I may need to check some configs with vite. The npm installs work but vite throws an error.

[abmusse]

** update I may need to check some configs with vite. The npm installs work but vite throws an error.

@bskimball

Interesting, Is there a simple example project that uses esbuild? Would like to use it to test using esbuild-wasm on IBM i.

[bskimball]

** update I may need to check some configs with vite. The npm installs work but vite throws an error.

@bskimball

Interesting, Is there a simple example project that uses esbuild? Would like to use it to test using esbuild-wasm on IBM i.

I was using @fastify/vite but I think remix with the vite package may be easier to set up to test SSR with vite and is here.

I haven't dug into much but it looks like even with the overrides in package.json vite tries to pull esbuild instead of esbuild-wasm. We host a bunch of IBMi servers, so I was going to screw around with it when I got a chance.

Probably related to this issue

[abmusse]

@evanw

Would it make sense to install esbuild-wasm as an optional dep when the os is IBM i (os400)?

esbuild/npm/esbuild/package.json

Line 20 in 9f9e4f8

"optionalDependencies": {

Node.js is available and could be used to run the wasm binaries.

This would install esbuild-wasm when packages require esbuild e.g. Vite in this case.

[merceyz]

esbuild has WASM fallbacks for certain platforms, for example https://github.com/evanw/esbuild/tree/40711afe0baa545012b813c5c7788225be9ef74c/npm/%40esbuild/android-x64, so that is an option.

[abmusse]

Yes I think having that for IBM i would be good. That way we can fallback to esbuild-wasm when esbuild is installed. Before doing that though are there tests for esbuild-wasm that I can run to verify things are working properly?

[abmusse]

I'll see about running the tests and updating with the results!

[abmusse]

I cloned the repo and I tried to run the wasm-tests:

node scripts/wasm-tests.js

Error: spawn go ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:284:19)
    at onErrorNT (node:internal/child_process:477:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn go',
  path: 'go',
  spawnargs: [
    'build',
    '-o',
    '~/esbuild/npm/esbuild-wasm/esbuild.wasm',
    '-ldflags=-s -w',
    '-trimpath',
    '~/esbuild/cmd/esbuild'
  ],
  cmd: 'go build -o ~/esbuild/npm/esbuild-wasm/esbuild.wasm -ldflags=-s -w -trimpath ~/esbuild/cmd/esbuild'
}

Looks like that failed trying to build the wasm lib with go.

esbuild/scripts/wasm-tests.js

Lines 360 to 363 in 40711af

	async function main() {
	// Generate the WebAssembly module
	const esbuildPathNative = path.join(__dirname, '..', process.platform === 'win32' ? 'esbuild.exe' : 'esbuild');
	await buildWasmLib(esbuildPathNative);

esbuild/scripts/esbuild.js

Lines 129 to 141 in 40711af

	exports.buildWasmLib = async (esbuildPath) => {
	// Asynchronously start building the WebAssembly module
	const npmWasmDir = path.join(repoDir, 'npm', 'esbuild-wasm')
	const goBuildPromise = new Promise((resolve, reject) => childProcess.execFile('go',
	[
	'build',
	'-o', path.join(npmWasmDir, 'esbuild.wasm'),
	'-ldflags=-s -w', // This removes ~0.14mb of unnecessary WebAssembly code
	'-trimpath',
	path.join(repoDir, 'cmd', 'esbuild'),
	],
	{ cwd: repoDir, stdio: 'inherit', env: { ...process.env, GOOS: 'js', GOARCH: 'wasm' } },
	err => err ? reject(err) : resolve()))

We don't have go available to build the wasm lib from.

Are there any alternatives of using a pre-built wasm lib?

[bskimball]

@abmusse I got back around to this and set up a test remix app that uses vite. The overrides in package.json does in fact use the wasm version. However, there are other errors showing in the wasm-function:

RuntimeError: float unrepresentable in integer range
    at wasm://wasm/02bc523a:wasm-function[1066]:0xe5a2c
    at wasm://wasm/02bc523a:wasm-function[1476]:0x147f82
    at wasm://wasm/02bc523a:wasm-function[743]:0xa740a
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[1061]:0xe594e
    at wasm://wasm/02bc523a:wasm-function[1059]:0xe5921
    at globalThis.Go.run (/var/apps/remix-test/my-remix-app/node_modules/esbuild/wasm_exec.js:527:23)
failed to load config from /var/apps/remix-test/my-remix-app/vite.config.js
Error: The service was stopped
    at /var/apps/remix-test/my-remix-app/node_modules/esbuild/lib/main.js:1074:25
    at responseCallbacks.<computed> (/var/apps/remix-test/my-remix-app/node_modules/esbuild/lib/main.js:694:9)
    at Socket.afterClose (/var/apps/remix-test/my-remix-app/node_modules/esbuild/lib/main.js:684:28)
    at Socket.emit (node:events:539:35)
    at endReadableNT (node:internal/streams/readable:1344:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

I don't know that falling back to esbuild-wasm would resolve anything.

side note: I hope IBM figures out a way to allow us to run Go on the IBMi platform.

[bskimball]

@abmusse this library https://github.com/onlysumitg/ibmigo/tree/main does run and compile Go on IBMi

[abmusse]

@abmusse I got back around to this and set up a test remix app that uses vite. The overrides in package.json does in fact use the wasm version. However, there are other errors showing in the wasm-function:

RuntimeError: float unrepresentable in integer range
    at wasm://wasm/02bc523a:wasm-function[1066]:0xe5a2c
    at wasm://wasm/02bc523a:wasm-function[1476]:0x147f82
    at wasm://wasm/02bc523a:wasm-function[743]:0xa740a
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[743]:0xa7277
    at wasm://wasm/02bc523a:wasm-function[1061]:0xe594e
    at wasm://wasm/02bc523a:wasm-function[1059]:0xe5921
    at globalThis.Go.run (/var/apps/remix-test/my-remix-app/node_modules/esbuild/wasm_exec.js:527:23)
failed to load config from /var/apps/remix-test/my-remix-app/vite.config.js
Error: The service was stopped
    at /var/apps/remix-test/my-remix-app/node_modules/esbuild/lib/main.js:1074:25
    at responseCallbacks.<computed> (/var/apps/remix-test/my-remix-app/node_modules/esbuild/lib/main.js:694:9)
    at Socket.afterClose (/var/apps/remix-test/my-remix-app/node_modules/esbuild/lib/main.js:684:28)
    at Socket.emit (node:events:539:35)
    at endReadableNT (node:internal/streams/readable:1344:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

I don't know that falling back to esbuild-wasm would resolve anything.

side note: I hope IBM figures out a way to allow us to run Go on the IBMi platform.

Ah so falling back to esbuild-wasm is still not ideal 😞

[phpdave]

I'm also having troubles using Vite which is using esbuild dependency on IBMi (os400).

When i run this

npm create vite@latest
cd vite-project/

modify the package.json to use wasm

"overrides": {
    "vite": {
      "rollup": "npm:@rollup/wasm-node@*"
    },
    "rollup": "npm:@rollup/wasm-node@*",
    "esbuild": "npm:esbuild-wasm@latest"
  }

I'm getting this panic: assignment to entry in nil map

npm install
npm run dev
bash-5.2$ npm run dev

> [email protected] dev
> vite --port 8765


  VITE v5.2.10  ready in 1082 ms
bash-5.2$ panic: assignment to entry in nil map

goroutine 9 [running]:
github.com/evanw/esbuild/internal/config.ProcessDefines(0x891288)
        github.com/evanw/esbuild/internal/config/globals.go:967 +0x52
github.com/evanw/esbuild/pkg/api.validateDefines({0x8c0d20, 0x80a330, 0x80a348, 0x80a360, 0x3, 0x8c0b40}, 0x8c0540, {0x0, 0x0, 0x0}, ...)
        github.com/evanw/esbuild/pkg/api/api_impl.go:668 +0xba
github.com/evanw/esbuild/pkg/api.validateBuildOptions({0x2, 0x0, 0x0, 0x8c05d0, 0x0, {0x0, 0x0}, 0x0, 0x0, {0x0, ...}, ...}, ...)
        github.com/evanw/esbuild/pkg/api/api_impl.go:1212 +0x18
github.com/evanw/esbuild/pkg/api.contextImpl({0x2, 0x0, 0x0, 0x8c05d0, 0x0, {0x0, 0x0}, 0x0, 0x0, {0x0, ...}, ...})
        github.com/evanw/esbuild/pkg/api/api_impl.go:879 +0x53
github.com/evanw/esbuild/pkg/api.Context({0x2, 0x0, 0x0, 0x8c05d0, 0x0, {0x0, 0x0}, 0x0, 0x0, {0x0, ...}, ...})
        github.com/evanw/esbuild/pkg/api/api.go:525 +0x2
main.(*serviceType).handleBuildRequest(0x8c00c0, 0x0, 0x8c01b0)
        github.com/evanw/esbuild/cmd/esbuild/service.go:739 +0xa0
main.(*serviceType).handleIncomingPacket.func2()
        github.com/evanw/esbuild/cmd/esbuild/service.go:235 +0x4
created by main.(*serviceType).handleIncomingPacket
        github.com/evanw/esbuild/cmd/esbuild/service.go:233 +0x27panic: assignment to entry in nil mapnpm ls esbuild

which is referencing

esbuild/internal/config/globals.go

Lines 967 to 969 in 70893e4

	result.IdentifierDefines["NaN"] = DefineData{
	DefineExpr: &DefineExpr{Constant: &js_ast.ENumber{Value: math.NaN()}},
	}

Is this also something related to WASM causing the issue?

node -p '[process.platform, os.arch(), os.endianness()]'
[ 'os400', 'ppc64', 'BE' ]

I followed the stack trace but still cant figure it out