plasma-umass / browsix Goto Github PK

What would be options for GUI support in browsix ?

It looks like a branch of Qt is being developped for WebAssembly compatibility (see http://qtandeverything.blogspot.fr/2017/06/qt-for-web-assembly.html and https://bugreports.qt.io/browse/QTBUG-63917 )

Also, maybe the broadway Gdk backend could be integrated (HTML5 canvas) offering some Gtk support, although that seems more complex, as Gtk static compiling may no longer be possible (see https://mail.gnome.org/archives/gtk-list/2018-January/msg00006.html )

Have you investigated such ideas ?

We currently have a few typescript errors like:

src/kernel/kernel.ts(16,28): error TS2307: Cannot find module './http_parser'.

This is due to the way I've mixed in JS from the node stdlib with our own typescript -- the TypeScript compiler can't see the module, but it is present during the browserify run. I think the new TypeScript feature that lets you pass JS through the typescript compiler will fix this.

Hi, guys,
I'm a newbee who wants to participate this project in GSoC. I want to implement mv command as my starter task.

sh currently takes a shell pipeline as an argument, executes the pipe, and returns results when all children have exited. This needs to be expanded so that it can run in interactive mode, potentially proxying input from the user to an active child process. This will also require changes to the kernel APIs for incremental input + output to a task

I mean. on main page, PLEASE write how to port/build/write programs to this OS. Should I run some compiler ? Can I write in C, or I must use JS as a source language ? I've read everything, but still does not understand how to make hello_world application from ground.

We should implement time as a command. This is sometimes implemented as a shell builtin in addition, but lets begin with a standalone program.

One thing to think about is how time should get and display information on how long it took a child to run. The simplest thing would be to record the time at which we issued the spawn syscall to the time we received SIGCHLD. It might be better tho to extend the kernel to keep track of this info, and implement a standard syscall like getrusage to access that info from time.

If I install node 4.3.0 as recommended, I get the following error:

/Users/aseering/browsix/node_modules/tslint/node_modules/@babel/code-frame/lib/index.js:39
  const {
        ^

SyntaxError: Unexpected token {
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:374:25)
    at Object.Module._extensions..js (module.js:417:10)
    at Module.load (module.js:344:32)
    at Function.Module._load (module.js:301:12)
    at Module.require (module.js:354:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/Users/aseering/browsix/node_modules/tslint/lib/formatters/codeFrameFormatter.js:20:20)
    at Module._compile (module.js:410:26)
    at Object.Module._extensions..js (module.js:417:10)
make: *** [test-once] Error 1

According to this StackOverflow post, this syntax isn't correctly supported by node 4.3; it requires a newer version.

If I install node 12, I get a different error:

fs.js:34
} = primordials;
    ^

ReferenceError: primordials is not defined
    at fs.js:34:5
    at req_ (/Users/aseering/browsix/node_modules/natives/index.js:143:24)
    at Object.req [as require] (/Users/aseering/browsix/node_modules/natives/index.js:55:10)
    at Object.<anonymous> (/Users/aseering/browsix/node_modules/vinyl-fs/node_modules/graceful-fs/fs.js:1:37)
    at Module._compile (internal/modules/cjs/loader.js:1139:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1159:10)
    at Module.load (internal/modules/cjs/loader.js:988:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1028:19)
    at require (internal/modules/cjs/helpers.js:72:18)

According to this StackOverflow post, this is a compatibility issue with gulp 3.x and newer versions of node. I tried bumping to gulp 4.0.2, but that broke some other bits of code; I didn't dig in further.

node 6.17.1 (the latest in the node 6.x line), however, seems to work just fine.

We currently deliver the SIGCHLD signal to parents when a spawn'ed process dies. We should implement the kill(1) command.

This will require adding a new kill(2) syscall, and extending the kernel's kill function to not only terminate (in the case of SIGKILL), but dispatch a signal (as well as implementing the command kill that exercises this syscall.

xargs(1) takes input from stdin and passes it as the argv vector to another command.

This should be pretty straightforward, without the need for new syscalls.

What is the current status of support for executables in webassembly rather than asm.js? The most recent commit I can find related to it is ba4d26c. But #63 seems to suggest support for webassembly is currently in development, and this paper on arXiv talks about something called "browsix-wasm".

I don't see a branch or another repository that seems like it would correspond to that "browsix-wasm". Does this mean it exists (under development) but the code isn't (yet) publicly released for some reason?

Hi.

I've been following instructions to try and build and use the Docker container.

I've built an image for browsix:a17538c, but the run fails on:

# make serve
  SERVE
[05:20:18] Using gulpfile ~/browsix/gulpfile.js
[05:20:18] Starting 'copy-node-kernel'...
[05:20:18] Starting 'copy-node'...
[05:20:18] Starting 'lint-kernel'...
[05:20:18] Starting 'lint-browser-node'...
[05:20:18] Starting 'lint-bin'...
[05:20:18] Starting 'lint-syscall-api'...
[05:20:18] Starting 'app:styles'...
[05:20:18] Starting 'app:images'...
[05:20:18] Finished 'copy-node-kernel' after 653 ms

events.js:141
      throw er; // Unhandled 'error' event
      ^
TypeError: this is not a typed array.
    at Function.from (native)
    at /root/browsix/node_modules/gulp-autoprefixer/index.js:25:27
    at run (/root/browsix/node_modules/karma/node_modules/core-js/modules/es6.promise.js:66:22)
    at /root/browsix/node_modules/karma/node_modules/core-js/modules/es6.promise.js:79:30
    at flush (/root/browsix/node_modules/karma/node_modules/core-js/modules/_microtask.js:18:9)
    at nextTickCallbackWith0Args (node.js:419:9)
    at process._tickDomainCallback (node.js:389:13)
make: *** [serve] Error 1

I have no clue what mess can lead to that error, sorry.

Is there a known tagged Git revision that's reputed working ?

Thanks in advance.

Hey there.

This project looks amazing, and is exactly like what I'm looking for. In fact, I'm trying to create an in browser web terminal much like your example, but one that includes the truffle command line tool for blockchain development.

From what I've read in the readme, I don't see a clear way to:

• Set up a webpage to use the kernel
• Run a nodejs application within the terminal/kernel
• Giggle like a schoolgirl at how awesome this is

Effectively, I need to create your demo with one other command line executable included. Can you tell me what I need to do? You might need to dumb it down for new users like me, aka, "add this script file to the , then compile your node app with browsix, include that script file, etc.". I feel like I'm missing some really obvious documentation, honestly. Being able to use browsix would be so amazing for me.

Some things I'd be looking to do once implemented:

1. Examine the filesystem in browser, outside of the kernel (say, in local storage) so files can be downloaded by the user.
2. Integrate it all with React (I'm happy to figure that one out)

Any help you can provide would be amazing!

set up CI - probably travis

Find is super useful. Full documentation is here. For our implementation we need at least the following flags implemented:

• -name $NAME
• -iname $NAME
• -type [df]

For details on these flags, check the man page above.

Here is a fun one.

• Go to https://unix.bpowers.net/
• enter any bad command (try cd or dfgug)
• Hit backspace/delete twice (repeat for recursive errors!)

When using lib-dist/lib/kernel.js from [email protected], I am no longer seeing stdout updates. My project is a fork of the LaTeX demo, and I'm making a pdflatex call through kernel.system(cmd, runNext, onStdout, onStderr).

If it helps, I'd be happy to create a standalone demo to recreate the error. Thanks!

I don't think it is necessary, and it seems to be causing people errors.

It is currently impossible to access https://browsix.org/ or https://unix.bpowers.net/ because of an expired SSL certificate.

It currently fails with ENOENT, but it should actually leave the shell unusable.

Deleting .deletedFiles.log causes OverlayFS problems. The file is removed, but internally it seems to assume it is never removed. Revisit this when we've updated to latest BrowserFS to see if it is just fixed upstream.

we need to walk the given path asynchronously, creating the directories we need along the way. Look at sbase or openbsd for inspiration.

The node executable in the browsix example hosted here throws an error Path must be a string. Received undefined.

Steps to reproduce:

1. Visit https://unix.bpowers.net/
2. Run node. Note the following error in the Chrome JavaScript console:

Uncaught TypeError: Path must be a string. Received undefined
    at assertPath (kernel.js:447)
    at Object.resolve (kernel.js:479)
    at Syscalls.open (kernel.js:2050)
    at AsyncSyscalls.open (kernel.js:1699)
    at Kernel.doSyscall (kernel.js:2469)
    at Task.syscallHandler (kernel.js:2871)

Note that this also breaks the shell, which stops responding to user input.

3. Run node -e 'console.log("hello")'. Note the following error appears in the bash console: error: -2

Hello, first of all kudos to the plasma team! I am amazed by your great work!
I was wondering if you could help me out with setting up the latex example. I managed to run the console, but I feel a bit confused as to running the latex example. I have no idea where to start or what to do in order to make it run. Further, how would it be possible to integrate browsix into a non-nodejs project?

Best,
Marko

Currently, dash is a checked-in artifact built on my machine. The issue is that it needs a custom emscripten toolchain with Browsix support, and that is a much larger dependency to pull in compared to simply node + npm + Make.

curl always fails, even when using unix.bpowers.net to get around same origin policy.

$ curl https://unix.bpowers.net 
Error while executing undefined: Error: socket hang up

I'm not sure if this was intended, but the build files in lib/ are not in the published npm package. There is a prepublish script, and I believe npm is ignoring the generated lib/ files on publish, because it by default ignores all files in .gitignore.

My use case is I'm trying to use browsix in a web app and installed browsix as an npm dependency. Thanks!

Is it possible not to rely on Node after initial setup? If so, how? Thanks, this project looks awesome. And, it's also possible that once I accomplish what I want to do with it that I may contribute back.

I've rebuilt a Docker container image based on 0.9.2, which builds ok.

During "make serve" execution inside the container, there's a weird prompt "Answer" caused by a npm prompt which is hidden during the bower_components target.

I think that the following patch would help:

@@ -70,7 +70,7 @@ $(NPM_DEPS): node_modules
 
 bower_components: $(BOWER) bower.json
 	@echo "  BOWER"
-	$(BOWER) install --silent
+	$(BOWER) install
 	touch -c $@
 
 syscall-api: $(BUILD_DEPS)

Of course, fixing the dependencies might be an alternative.

The dialog is actually the following:

Unable to find a suitable version for test-fixture, please choose one by typing one of the numbers below:
    1) test-fixture#^1.1.1 which resolved to 1.1.2 and is required by browsix
    2) test-fixture#^3.0.0 which resolved to 3.0.0 and is required by web-component-tester#6.4.1

Prefix the choice with ! to persist it to bower.json

? Answer 

When responding with first option, that seems to be working.

Hope this helps

When I last tried this, I hit issues with Browserify. Anyway, it is of course solvable somehow, so I'm leaving this here as an issue to come back to after paper deadline.

We currently handle node.js and Go applications differently: Go applications are compiled to JS along with a small, inline syscall shim that knows how to talk over the Worker's MessagePort. For node, we have a browser-node binary, that then loads and interprets javascript files. This works well for a demo, but is sub-par for non-toy applications that require() things outside of the stdlib.

There are 2 ways we could go with this that aren't mutually exclusive:

• use parts of Browserify (or one of the alternatives like webpack or rollup.js) in browser-node. When node ls.js gets executed, we read the ls.js file, parse it, find all the require statements, and transitively pull into the node process all depenencies before executing the script. We need to do this before we begin execution, because require() has blocking semantics.
• implement and document a Browserify invocation that replaces a bunch of the Browserify standard library imports with our own that talk over system calls. Then we can directly execute these 'binaries'.

The second one is probably better for all use cases except demos, where people might modify and re-execute node scripts.

I saw that most of the command files have their own command line parser.

This increases code duplication.

Making the Command line Parser as a module or importing a pre-implemented library would be better.
It will ensure code re-usability and easy scaling for future commands.

PS: If you like this proposal, I would like to take this up.

Don't know how this one slipped by, but of course we should have cp(1).

Most important flag: -R (recursive).

Right now, we return a string, but we should return a Buffer. This would better match existing node.js APIs, and the output of commands isn't guaranteed to be valid unicode, so we shouldn't be doing this on behalf of callers.

Is this still an active project? If i'm looking to run R code (and possibly some ggplot2 outputs) directly in the web browser, is this tool a possibility?

It is confusing that after you've started executing a command, you can't send additional input to the command's stdin. This causes things like

$ cat

To effectively hang the shell, even though they are behaving according to the specification. While we're at it, we should catch ctrl-c.

I have tried to understand whether I could run an HTTP server in my browser that others in my network (or even publicly) could access. I am unable to find this in clear terms in your documentation.

Will I be able to serve a static website from my browser to others in the web?

$ multiline commands\
/usr/bin/multiline: command not found
$ FOO="bar"
/usr/bin/FOO="bar": command not found
$ ls $(ls usr)
ls: /$(ls: No such file or directory.
ls: /usr): No such file or directory.
$ ls `ls usr`
ls: /`ls: No such file or directory.
ls: /usr`: No such file or directory
$ # comment
/usr/bin/#: command not found
$ echo hi > README
hi > README
$ echo hi >> README
hi >> README
$ echo foo && echo bar
foo && echo bar
$ echo foo; echo bar
foo; echo bar
$ ls *
ls: /*: No such file or directory.
$ echo !!
!!
$ usr/bin/echo foo
/usr/bin/usr/bin/echo: command not found
$ ./README
Error while executing undefined: SyntaxError: missing ; before statement

(The last one should fail, but because of permissions.)

as @jvilk pointed out in #1, jvilk/BrowserFS provides ApiErrors to callbacks, which have a pretty toString method. We lose this when we send these errors from the kernel to the worker (due to structured cloning "The prototype chain does not get walked and duplicated").

We should provide full ApiErrors to users of the fs module in browser-node. BrowserFS's WorkerFS does this - we should probably just use that same mechanism

So, I thought I'd take a shot at getting Browsix running on Windows 10 (the first one, summer 2015). Here's the issue as it happened (wish I was on that computer right now). I would put the debug file and the console log here, but it's stored on another computer at the moment.

The issue is when it goes to copy the user's install of node.js, and then apply mods to it. It looks like the same issue from 4.3.x to 7.0.x. The first issue is that it can't find "./http_parser." The next issue is that it can't find "./bin" and all the rest of the files presumably in the Gulpfile.

I was looking at a thread on GitHub that had a similar issue with http_parser across the board (mac, Linux, and Windows).

If anyone can help building with cygwin, please let me know! Thanks in advance.

on a normal unix-like OS, if you write to a pipe more than can be appended to its internal buffer, your write block until the other end of the pipe reads out the data. This provides a good system that avoid unbounded buffer growth in the kernel -- we should implement the same thing. We currently already have logic for having reads from a pipe block, this should be somewhat symmetric with that

Really interesting project. Maybe in the future if you want to implement startx command it would be really cool for me if you consider using https://github.com/fibo/dextop if it makes sense.

Hi there,

This project looks amazing!!!

I'd like to use it in order to create interactive documentation about the unix shell commands.

I was thinking of an integration of browsix into a tool of mine named klipse. In order to do that all I need is a standalone javascript file that exports the shell commands and outputs to the console.

Then we could have a html page with several code snippets that share the same environment and the reader would be able to play with the code and get a better understanding of the shell commands.

Does it make sense?
Do you think it's feasible?

I'm not exactly sure what the protocol should be, but @shivamkakkar brought up the good point that the error handling in spawn isn't complete. Right now we eagerly try to execute whatever the user wants, even if that thing ends up being a non-Browsix javascript or text file.

We should have a protocol whereby we launch a worker, send it the init message, and wait for a reply. If we don't get a reply in < $N milliseconds, we kill it and report an error. I'm open to other proposals, too.

I just came to know that pwd command is not implemented yet. I would like to contribute.

In my opinion npm could be the perfect package manager for browsix.

Abhishek Kumar reported the following error when running $ make test-once serve

  NPM
  CXX(target) Release/obj.target/pipe2/pipe2.o
  SOLINK_MODULE(target) Release/obj.target/pipe2.node
  SOLINK_MODULE(target) Release/obj.target/pipe2.node: Finished
  COPY Release/pipe2.node
  CXX(target) Release/obj.target/priority/priority.o
  SOLINK_MODULE(target) Release/obj.target/priority.node
  SOLINK_MODULE(target) Release/obj.target/priority.node: Finished
  COPY Release/priority.node
  CXX(target) Release/obj.target/WebWorkerThreads/src/WebWorkerThreads.o
  SOLINK_MODULE(target) Release/obj.target/WebWorkerThreads.node
  SOLINK_MODULE(target) Release/obj.target/WebWorkerThreads.node: Finished
  COPY Release/WebWorkerThreads.node
Makefile:58: recipe for target 'node_modules' failed
make: *** [node_modules] Error 3

Could you report the following info:

• The output of npm version.
• The output of uname -a.
• The output of running npm install.

Thanks Abhisek!