garden / tree Goto Github PK

I just started the server and used the webapp normally, when suddenly, the search function's $fs calls weren't answered, because the server had crashed.

This is the error message:

/home/jan/pro/tree/lib/fs.js:204
    this.driver.read(this.path, function(err, data) {
                ^
TypeError: Cannot call method 'read' of undefined
    at File.open (/home/jan/pro/tree/lib/fs.js:204:17)
    at /home/jan/pro/tree/lib/fsapi.js:23:12
    at Object.file (/home/jan/pro/tree/lib/fs.js:111:5)
    at read (/home/jan/pro/tree/lib/fsapi.js:16:6)
    at EventEmitter.fsOperation (/home/jan/pro/tree/lib/fsapi.js:95:7)
    at EventEmitter.emit (events.js:115:20)
    at IncomingMessage.gotrequest (/home/jan/pro/tree/node_modules/camp/lib/camp.js:188:12)
    at IncomingMessage.EventEmitter.emit (events.js:88:17)
    at IncomingMessage._emitData (http.js:359:10)
    at HTTPParser.parserOnBody [as onBody] (http.js:123:21)

Maybe it's a rare corner case, but even if it's hard to reproduce I think we can make our system more error resilient by adding a check.

Hi!

It would be really nice to create keynotes in tree.

David.

I need such a system to support many cool file types. Here are some examples I'd like to add:

• .rc, .bashrc, .vimrc, .bash_aliases, .bash_history... to text/x-sh
• .sequence to text/x-sequence
• .tex, .latex to text/x-latex
• .ml, .mli to text/x-ocaml
• .irc to text/x-irc
• a lot of the test files' mime types were set manually. I'll reset them and teach tree to guess those types back.

Hi,

The File Tree currently doesn't work on Windows.

Some metadata reads throw JSON syntax errors and crash the server. We should fix it.

While loading metadata for /alma : Error: JSON Syntax Error
    at /home/dom/tree/lib/driver.js:65:31
    at [object Object].<anonymous> (fs.js:123:5)
    at [object Object].emit (events.js:64:17)
    at Object.oncomplete (fs.js:1188:12)
[Error: JSON Syntax Error]

/home/dom/tree/lib/plug.js:156
            files[i].path + (files[i].isOfType('dir') ? '/' : ''));
                    ^
TypeError: Cannot read property 'path' of undefined
    at /home/dom/tree/lib/plug.js:156:21
    at /home/dom/tree/lib/fs.js:369:7
    at /home/dom/tree/node_modules/async/lib/async.js:190:13
    at /home/dom/tree/node_modules/async/lib/async.js:88:21
    at /home/dom/tree/node_modules/async/lib/async.js:187:17
    at /home/dom/tree/lib/fs.js:364:20
    at /home/dom/tree/lib/fs.js:130:9
    at /home/dom/tree/lib/driver.js:65:28
    at [object Object].<anonymous> (fs.js:123:5)
    at [object Object].emit (events.js:64:17)

I'm working on an android app for tree and I think the API need a few modifications:

• ls on a file return not only the file listing but also the content of every file in this directory, useless and a very bad point for mobile data payload

You could fix ls or add meta data to the directory listing with cat making cat a very powerful tool to navigate the tree.

• While accessing a file or a directory it would be great to have the last time updated in the http Header file making local cache validation easy and reducing data payload on a regular use case.

To see in action : http://jsbin.com/uwipin/2/edit

function toHex(number, length) {
  var max = Math.pow(16, length) - 1;
  if(number > max) number = max;
  if(number < 0) number = 0;
  var hex = number.toString(16);
  if(hex.length < length)
    hex = (new Array(length - hex.length + 1)).join('0') + hex;
  else
    hex = hex.substring(0, length);
  return hex;
}

function coloriser(chaine,seed)
{
  var nb=16777216;
  for (var i=0; i<chaine.length; i++) {
  nb = nb +chaine.charCodeAt(i);
  }
  nb = Math.pow(nb*nb+nb+seed,3);
  return toHex(nb % 0xFFFFFF + 1, 6);
}

/**
 * Pastelise la couleur rgb, avec une saturation et une lumière définie.
 */
function pastelize(rgb, saturation, light) {
    var channel, pastel, _i, _len, _ref;
    if(saturation === undefined) saturation = 128;
    if(light === undefined) light = 0;
    pastel = '';
    _ref = /(..)(..)(..)/.exec(rgb).slice(1);
    for (_i = 0, _len = _ref.length; _i < _len; _i++) {
      channel = parseInt(_ref[_i], 16) % saturation + (128 - saturation) / 2 + light;
      channel = Math.max(Math.min(channel, 255), 0);
      pastel += toHex(channel, 2);
    }
    return pastel;
  }

function colorizeBackground(text) {
  var color = pastelize(coloriser(text,seed), 150,100);

  document.bgColor = color;
}

var seed = 4000;

var moninput = document.getElementById("pouet");

moninput.addEventListener("keyup", function(e){
  colorizeBackground(moninput.value);
}, false);

colorizeBackground("");

And this one for some color test ( light change doesn't work here, still is 100)
http://jsbin.com/uwipin/8/edit

http://www.thefiletree.com should redirect to https://thefiletree.com

Hi,

I do a presentation to Tiago, and I catch this error

Thanks a lot !

Hi,

If you write on a file when the instance is on, there is no problem and the file is correctly kept in memory.

But when you leave the instance (Ctrl+C on the terminal on which you've launched the filetree with "make"), the content of the file may be lost.

Hi!

That would be nice to give a comment to someone that he receives an email
These comments could be a different color compared to the general comments

Thanks a lot!
David

In pencil.html and preview.html, the text editor only takes a portion of the height of the page. Someone should fix that.

Hi,

I tried to access a directory with the attribute ?plug=none, but I had an error.
Here is a example url : https://thefiletree.com/?plug=none

Here is the error displayed by firefox :
Erreur d'encodage de contenu (La page que vous essayez de voir ne peut être affichée car elle utilise un type de compression invalide ou non pris en charge.)

Hi,

Previously, I made an example of a proof-of-concept web app (a todo list app, of course), using thefiletree as a backend for storing data. This application was a terrible hack of the default comportment, as this app is just based on the pencil plugin, hiding all html content but the frontend of the web app.

As this hack is very dirty, would it be possible to change the API to be able to send modifications easily?

As far as I found, changes are sent by the CodeMirror client (which is a layer over the main textarea) which creates the operational transformations. In this case, it would be sufficient to split up the operational transformations management and the CodeMirror client. Is this correct?

Seems that the operation "apply" in fs.js should be implemented for this purpose too.

This is a new error I didn't have before. It seems to happen at random.

$ sudo node app

/home/jan/pro/tree/node_modules/camp/lib/camp.js:51
  this.uri = url.parse(decodeURI (req.url), true);
                       ^
URIError: URI malformed
    at new Ask (/home/jan/pro/tree/node_modules/camp/lib/camp.js:51:24)
    at Server.listener (/home/jan/pro/tree/node_modules/camp/lib/camp.js:121:13)
    at Server.<anonymous> (native)
    at Server.emit (events.js:70:17)
    at HTTPParser.onIncoming (http.js:1514:12)
    at HTTPParser.onHeadersComplete (http.js:102:31)
    at Socket.ondata (http.js:1410:22)
    at TCP.onread (net.js:354:27)

@espadrine if you can reproduce or fix the problem, thanks! :)

Running 35f5cc4 from espadrine
I can crash my local server reliably with

http://localhost:8082/robots.txt?plug=death

localhost:~/tmp/garden/tree$ tail -f tree.log
Error while getting the metadata of /death.html
Error: ENOENT, stat '/var/home/adrian/tmp/garden/tree/web/death.html'

/var/home/adrian/tmp/garden/tree/lib/fs.js:129
        delete fileFromPath[path];
                            ^
ReferenceError: path is not defined
    at /var/home/adrian/tmp/garden/tree/lib/fs.js:129:29
    at /var/home/adrian/tmp/garden/tree/lib/fs.js:168:18
    at Object.oncomplete (fs.js:297:15)

Was minding my own business, when this error happened, the server crashed and restarted and my file was wiped blank. Had to fetch backup to fix it (file was webjson.js). We should fix it and be way more robust to errors.

While loading metadata for /lib/js/webjson.js : Error: JSON Syntax Error
    at /home/dom/tree/lib/driver.js:65:31
    at [object Object].<anonymous> (fs.js:123:5)
    at [object Object].emit (events.js:64:17)
    at Object.oncomplete (fs.js:1190:12)
[Error: JSON Syntax Error]

http.js:536
    throw new Error("Can't set headers after they are sent.");
          ^
Error: Can't set headers after they are sent.
    at ServerResponse.setHeader (http.js:536:11)
    at end (/home/dom/tree/node_modules/camp/lib/camp.js:437:17)
    at /home/dom/tree/lib/plug.js:45:13
    at Object.file (/home/dom/tree/lib/fs.js:109:5)
    at end (/home/dom/tree/lib/plug.js:40:12)
    at /home/dom/tree/lib/plug.js:171:40
    at /home/dom/tree/lib/fs.js:288:7
    at /home/dom/tree/lib/fs.js:208:7
    at [object Object].<anonymous> (fs.js:123:5)
    at [object Object].emit (events.js:64:17)

When I try to open two tabs on localhost/root/jan/README.md, the first one loads the Pencil correctly templated, but the second one never loads (indifferently on chrome, rekonq and firefox).

See demo video : http://paul.cx/public/bug-tree.ogv

STR :

• Open a file in the file tree : https://thefiletree.com/root/humans.txt
• Try to select multiple line using the mouse, from somewhere inside the text.

Expected :

• Multiple lines can be selected

Result :

• A single line is selected.

Config

Firefox Nightly (16/12/2012), Linux (Ubuntu 11.10).

If I start the drag from under the text, it works fine.

Hi!

Is it possible to add a split screen feature in the pencil?

Thanks a lot!
David

Hi,
I may have over used Vim, but I've got naturel reflex, after having written something, to press Escape. After pressing escape, an assertion error shows up in Firefox and then, for every character written, we've got a wonderful "The base length of the second operation has to be the target length of the first operation" message that comes back. The only thing to do then is to reload the page.

I see a lot of these happening recently, seemingly at random. I was fighting to edit a file that refused to save, and when I ssh'd into the server to look at the logs it started working again. Strange...

fs:file: /jan/html/web_speech_synthesis.html count is negative (0). Keeping it at 0.
fs:write: Content not loaded or no write driver found.

Avoid in-html <script>s as if it was the devil.

Reason: try running the following URL in a browser. It should make sense pretty soon why <script> is evil.

data:text/html,<!doctype html><title></title><p>hello<script>alert("</script>")</script>

Accessing a file should be done with https://thefiletree.com/user/file instead of https://thefiletree.com/root/user/file. In order to achieve this, we need a way to specify how a file is accessed (eg raw, edition, preview, etc). This can be seen as a way to determine which plug to use.

I suggest two ways to specify a plug:

• Look into query string: https://thefiletree.com/user/file?plug=/util/pencil
• If not there, look into metadata (see #18)
• If not in file's metadata, look into parent's metadata (the topmost folder will specify the default values)

We would benefit from removing the contents of the current Makefile (which is optimized for Scout Camp development, and an old version with that), and starting over.

If I get a green light from at least one collaborator, I'll go ahead and do it.

@jankeromnes

Occasionally when pencil.html or preview.html loads, there is this error:

Uncaught TypeError: Object #<Object> has no method 'onnewcontent'
Scout.send.params.resp
Scoutmaker.sendxhr.xhr.onreadystatechange

Concerned lines are plugger.js:10 and scout.js:125.
I suppose the problem happens when the codemirror plug does not load fast enough.

Hi,

With the last version of tree, I have this error on TheFileTree.org
node.log file:

undefined:0
^
SyntaxError: Unexpected end of input
    at Object.parse (native)
    at /home/dom/tree/lib/driver.js:64:26
    at [object Object].<anonymous> (fs.js:115:5)
    at [object Object].emit (events.js:64:17)
    at afterRead (fs.js:1117:12)
    at Object.wrapper [as oncomplete] (fs.js:254:17)
CA file not found: https.ca
info: socket.io started

Currently, going to a page that does not exist in /root/ crashed the server.

Example: https://thefiletree.com/root/fail

I sometimes randomly get the following server error when I do a lot of text modifications on pencil / preview:

Delta length (383) does not equal source text length (387).
Trace: Error: Delta length (383) does not equal source text length (387).
    at Object.addnewstuff [as new] (/home/jan/pro/tree/server.js:192:51)
    at IncomingMessage.<anonymous> (/home/jan/pro/tree/lib/camp.js:192:57)
    at IncomingMessage.emit (events.js:67:17)
    at HTTPParser.onBody (http2.js:121:23)
    at Socket.ondata (http2.js:1347:22)
    at TCP.onread (net_uv.js:309:27)

The field date on a directory is updated every time we call read on it.
This problem doesn't seem to exist with regular files.

A lot of browsers are throwing a security exception with our certificate. We use the same cert provider as https://klaim.it, but their cert is always valid.

We should figure out why we run into trouble with ours, and fix it.

Minification seems broken with jsmin.

In Makefile, doesn't $chosenjsmin < "${file}" > "${file}min"; give the file name to $chosenjsmin? I believe jsmin takes javascript code instead of a file name.

This warning seems to happen a lot:

TYPE:isCompatible: type undefined (null) does not have fallbacks

Everything keeps running smoothly, but maybe we should investigate why this happens. Aren't all files supposed to have non-null types? Who tried to check compatibility on undefined types?

Currently, the visual block path in the gateway is too slow. It waits for the page to load, and a path change rebuilds all the blocks. This can certainly be made more efficient!

Apparently, the file system can lose a reference to folders.

For example, the epfl/information-theory/exercise1.md/ file that I created the other day is actually there on disk (in the server) but is not listed in the gateway.

Furthermore, running https://thefiletree.com/$fs?op=read&path=/ shows that the system itself does not see that file.

Obviously, restarting the server will make it aware of that file yet again, but @jankeromnes, don't do it, I'd like to debug it while I can.

1. while the server is running, open a text file in your browser, with the meta plug
2. set a metadata property (for instance, foo: "something") and save
3. open a new tab to edit the file with the pencil, make a few changes and close the tab
4. refresh the meta plug tab, and observe that the metadata is reset

The CodeMirror bundle we use in web/pencil.html contains a test suite for the xmlpure mode which uses jQuery. We do not have jQuery, so the browser throws an error.

Future CodeMirror bundles shouldn't contain this test suite.

The security information needed for a file are:

• meta key (master key allowing to edit metadata and keys)
• write key (for write protected files that are still readable)
• read key (for complete high-security data encryption)

Keys should be stored in metadata as salted Bcrypt hashes.

Hi!

When you have a big document with the pencil and you do a verification, it's good to lock this part (to not edit).

Regards,
David.

In order to allow users to compile / run / debug in any language, we need an efficient and secure sandboxing. When possible we should compile / run / debug client-side, and when not we should have server-side sandboxes ready.

Just loaded a text file on thefiletree.com

• Total loading time of the page: 8.57 seconds (lots of things loaded from cache)
• Loading time for socket.io.js: 8.42 seconds (not from cache, and most of it is transfer because latency was only 450ms)

Let's save 8.42 seconds (98.25%) of loading time by including socket.io.js in garden/plugs without using the weird $socket.io hack.

In the Gateway, I tried the input /j. I got this server-side error:

{ [Error: ENOENT, Success '/home/jan/pro/tree/root/j'] errno: 31, code: 'ENOENT', path: '/home/jan/pro/tree/root/j' }

/home/jan/pro/tree/server.js:34
        dir.content (function (err, content) {
            ^
TypeError: Cannot call method 'content' of undefined
    at /home/jan/pro/tree/server.js:34:13
    at Object.oncomplete (/home/jan/pro/tree/fs.js:197:7)
make: *** [start] Error 1

It seems to me that it means 'file not found'. In this case, we should fix that (calling a non-existing file should not crash the server).

For example, when you modify a html file and you press the "run" button, it opens a new windows without your latest modifications.
As a workaround, I press F5 before to press "run", and it works as expected.

Just to let you know that the server was down a long time, and the reason it crashed (and still crashes now) is this bug:

/home/dom/tree/lib/fs.js:201
    this.driver.read(this.path, function(err, data) {
                ^
TypeError: Cannot call method 'read' of undefined
    at File.open (/home/dom/tree/lib/fs.js:201:17)
    at /home/dom/tree/lib/fsapi.js:26:12
    at Object.file (/home/dom/tree/lib/fs.js:108:5)
    at read (/home/dom/tree/lib/fsapi.js:19:6)
    at EventEmitter.fsOperation (/home/dom/tree/lib/fsapi.js:108:7)
    at EventEmitter.emit (events.js:115:20)
    at IncomingMessage.gotrequest (/home/dom/tree/node_modules/camp/lib/camp.js:188:12)
    at IncomingMessage.EventEmitter.emit (events.js:88:17)
    at IncomingMessage._emitData (http.js:359:10)
    at HTTPParser.parserOnBody [as onBody] (http.js:123:21)

Hope we can fix it soon :)

The "collaborative" part wouldn't be very useful, but it would be awesome
to have a client distribution, that is,
a tree distribution that lets you interact with your own file system,
and write files wherever you want.

"Yo dawgs, I heard you liked OSes, so I built an OS in your OS so that you can F11 all day long!"

Files and metadata are written on disk as root. This is horrible. We should fix this. Administrator privileges should only be used on what they're asked for, namely listening on a port < 1024.

Side note: The title of the issue is a reference to EFF's new coder's rights sticker https://eff.org/r.3abJ

Some malicious users created files on thefiletree.com called can I code? and 笑嘻嘻 (yes there are two spaces in that name EDIT: even if markdown hides them). These files show up on the list but a click throws a 404 error.

LET'S FIX THIS GUYS

This is worst on slow connections. The arrow cursor in the Gateway is one of the last elements to load, but it's the one we'd expect to see first.

The camp server should gzip all the files, right? Here is an audit of /:

Compressing the following resources with gzip could reduce their transfer size by about two thirds (~30.39KB):
  http://localhost/ could save ~2.24KB
  common.css could save ~7.13KB
  gateway.css could save ~525B
  scout.js could save ~14.52KB
  gateway.js could save ~5.98KB

Currently the fuzzy matching search in the Gateway goes haywire. We should fix this!

Example : https://thefiletree.com/root/?q=some+text (also, it would be cool if links like this actually worked, probably related to #19)