nodesolidserver / node-solid-server Goto Github PK

I think the only correctly implemented test is test/params, see line
https://github.com/linkeddata/ldnode/blob/master/test/params.js#L70

supertest allow us to just test an express app, without running it on a port.
Running each test on a port is not just time consuming, but also, whenever two tests run on the same port, sometimes we get errors on travis.

This (with rdfACL control) is one of the two most pressing things for this project at the moment.

@deiu notices that @magnetik had actually done a webid for node https://github.com/magnetik/node-webid so we forked it here in linkeddata and it remains to integrate it with the rest of the server. Any volunteers?

TypeError: undefined is not a function
at Object.exports.publishDelta (/devel/github.com/linkeddata/ldnode/subscription.js:147:5)
at /devel/github.com/linkeddata/ldnode/handlers/patch.js:123:34
at doPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4340:9)
at $rdf.IndexedFormula.applyPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4369:9)
at handler (/devel/github.com/linkeddata/ldnode/handlers/patch.js:108:22)
at Layer.handle as handle_request
at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:112:3)
at Layer.handle as handle_request
at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:277:22

Same as any other suffix, suffixAcl, etc..

Underscore is an awesome library, but I was wondering if we really need it.

I was having a look at https://github.com/linkeddata/ldnode/blob/master/acl.js and I have seen that we are using _.each, which is the equivalent of obj.forEach that I believe it is also more readable.

Is there any reason for this particular choice?

While it could be useful to hard code the test folder, it should not be hard-coded in this way:

/devel/github.com/linkeddata/node-ldp-httpd/test/

since we are assuming the developer is storing code in this way :(

SoLiD apps require the presence of two Link headers in order to find where the corresponding ACL and meta resources are. The SoLiD document has examples here: https://github.com/linkeddata/SoLiD/blob/master/README.md#the-head-method

• test amount of listinting to equal readdir
• test File can't infer Container type
• test all files have File and all folders have Container type

Most of the function in parse.js implement what $rdf.parse is for, correct?

If not, this is a very valuable too and it should be separated in a different lib linkeddata-parse / ldnode-parse

We need the travis config file

https://travis-ci.org/linkeddata/ldnode/builds/67748220

A nine syllable name for this project is pretty painful. Whenever it comes up in conversation I find myself unable to remember the name, and unsure my listeners would understand/parse/recognize/remember it.

I suggest ldnode or ld-node would be a good name, along the lines of ldphp and ldpy, and not far from gold (which is ldgo re-ordered).

There are two solution for this:

1. Passed as arguments
2. Have them as global variables (e.g. process.env.URI_BASE)

I would prefer the second one having a fallback on the default test folder

exports.something >> module.exports.something

Following this and many other sources, unless we are exporting a function as the only thing we export (instead of an object).

Hoisting

Mentioned here! #21

Exports declared at the end

In some part of the code (e.g. https://github.com/linkeddata/ldnode/blob/master/acl.js) I have seen that we declare some exports.something, and then we use them as exports.something. I think that makes the code a bit unreadable.

I think declaring an export in the following way is more of a standard in the js community.

function Handler () {..
// and at the end of the file
exports.handler = Handler

PS: I was going to make some pull reqs, but I think we should first agree on this!

Usage of proxy should be added in command line and checked if it works

I am not sure where that comes from

This error I thought was fixed yesterday but has come back Monday morning 7/13.

ldnode:handlers PATCH -- text length: 342 +0ms
TypeError: Arguments to path.join must be strings
at path.js:360:15
at Array.filter (native)
at Object.exports.join (path.js:358:36)
at Object.uriToFilename (/devel/github.com/linkeddata/ldnode/fileStore.js:9:25)
at handler (/devel/github.com/linkeddata/ldnode/handlers/patch.js:18:25)
at Layer.handle as handle_request
at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:112:3)
at Layer.handle as handle_request
at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:277:22

timbl: > [email protected] postinstall /devel/github.com/linkeddata/ldnode/node_modules/webid

> cake build
sh: cake: command not found

This was trying to do an install from scratch of ldnode

Would it be possible to add the coffee-script dependency to the install?

  Link:     <http://www.w3.org/ns/ldp#Resource>; rel="type", </timbl/2015/08/02/poll/forms.ttl.changes>; rel="changes", <https://timbl.com/timbl/2015/08/02/poll/forms.ttl.acl>; rel="acl", <https://timbl.com/timbl/2015/08/02/poll/forms.ttl.meta>; rel="describedBy"

should be

  Link:     <http://www.w3.org/ns/ldp#Resource>; rel="type", <forms.ttl.changes>; rel="changes", <forms.ttl.acl>; rel="acl", <forms.ttl.meta>; rel="describedBy"

The Solid world requires the User: header to be sent back when a user is logged on with webid/TLS

Typical response headers from a GET authenitcated by webid currently:

status 200
status Text OK
connection keep-alive
content length 1367428
content type text/n3; charset=utf-8
etag W/"14dd84-P7TE3CNZwvMq1mvb6/zJ+w"
link http://www.w3.org/ns/ldp#Resource; rel='type', </TimBL/Automation/mother/state.n3.changes>; rel='changes'
ms author via SPARQL
updates via /TimBL/Automation/mother/state.n3.changes
x powered by Express
x response time 4.160ms

ldnode does not currently support CORS headers. We need to have at least Access-Control-Allow-Origin support.

At the moment I am trying to run the tests but they all fail since they can't find the resources, hence the socket hang-up.

(I know they are under development! This is just to know how they work!)

• The test folder should be cleaned, meaning that all of its files that are not .js should be in a folder resources and testfiles should be a subdirectory of resources (it is important for testing to have a subdir!), possibly called testContainer. (done #80)
• Splitting the test.js and restructure tests (done #85)
• Also, there are files that are not needed.

Running ldnode -v no longer outputs debug information to stdout.

Following the LinkedData 2nd meeting. There is no need to set up the baseUri.

This should be set up by looking at the host in the request.

Remember: Always use the base that you find in the request

In case it is not specified, use process.cwd()

This is caused by tls.getPeerCertificate() getting a wrong modulus. This is the case of @deiu's webId: https://deiu.me/profile#me

Code can be taken and adapted from gold: https://github.com/linkeddata/gold/blob/master/server.go#L884

This is the last bit that's missing from ldnode, which would make it fully compatible with warp.

stderr exert:
Maybe the debug() function is undefined
This happens after a successful PATCH has been serviced, in the code to publish the change subscriptions. Whether or not --no-live is set in the command live

Tue, 14 Jul 2015 15:35:12 GMT ldnode:handlers PATCH -- applied OK (sync)
TypeError: undefined is not a function
    at Object.exports.publishDelta (/devel/github.com/linkeddata/ldnode/subscription.js:147:5)
    at /devel/github.com/linkeddata/ldnode/handlers/patch.js:123:34
    at doPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4340:9)
    at $rdf.IndexedFormula.applyPatch (/devel/github.com/linkeddata/ldnode/node_modules/rdflib/dist/rdflib.js:4369:9)
    at handler (/devel/github.com/linkeddata/ldnode/handlers/patch.js:108:22)
    at Layer.handle [as handle_request] (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/layer.js:95:5)
    at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:131:13)
    at Route.dispatch (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/layer.js:95:5)
    at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:277:22

I need to be able to set up a custom return HTML page for HTTP errors, especially error 401 "Unauthenticated". In practice this is an important part of the on-boarding process, to lead someone who does not have a webid to grab one.

Apache does this with the ErrorDocument directive a la
ErrorDocument 404 /Lame_excuses/not_found.html
ErrorDocument 401 /Subscription/how_to_subscribe.html

See https://httpd.apache.org/docs/2.2/custom-error.html

This should be added in the repo, package.json and README

Tests are too slow, we should use nock to mock HTTP requests and speed up the tests, especially ACL

The idea is to use routes instead of implementing it straight into the main toutes

// in server.js or whoever will implement it
app.use('/what_i_desire', routes);

// in our library
var express = require('express');
var router = express.Router();

/* GET home page. */
router.get('*', function(req, res) {
});

router.patch('*', function(req, res) {
});

module.exports = router;

In this way, we can make something that any user can plug and play in their environment

This is only available in the latest express. Good idea to upgrade?

debug trace, filtered:

ldnode:ACL Looking for policies in /Library/WebServer/Documents/TimBL/Automation/Library/Mashup/jquery.min.js.acl +0ms
ldnode:ACL Looking for policies in /Library/WebServer/Documents/TimBL/Automation/Library/Mashup/.acl +0ms
ldnode:ACL Looking for policies in /Library/WebServer/Documents/TimBL/Automation/.acl +0ms
ldnode:ACL Looking for policies in /Library/WebServer/Documents/.acl +0ms
ldnode:ACL Found policies in /Library/WebServer/Documents/.acl +0ms

It never found the one in /Library/WebServer/Documents/TimBL/Automation/Library/.acl

with
commit 30dc1b5
Author: nicola [email protected]
Date: Mon Jul 13 00:45:33 2015 +0200

It looks like dirname() being invoked twice ...

I was wondering why every single option would be needed

 app.use(session({
    secret: 'node-ldp',
    saveUninitialized: false,
    resave: false
  }));

I consider the secret: 'node-ldp' by default a serious security issue

Still present here https://github.com/linkeddata/ldnode/blob/master/handlers/get.js#L20

The tests that we have in our folder are acceptance tests, in other words we are testing if the API works as expected. The same tests can be used for any other LDP implementation, e.g. gold.

So, my idea is that we could just take this tests out in a new package like ld-test or superld that tests a LDP. So that in this repo we can test more on unit testing.

This is just an idea, otherwise we could just leave things as they are, which are great!

https://github.com/linkeddata/ldnode/blob/master/handlers/get.js#L21

Gold (or databox.me) returns a listing of this kind:

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

<>
    a <http://www.w3.org/ns/ldp#BasicContainer>, <http://www.w3.org/ns/ldp#Container>, <http://www.w3.org/ns/posix/stat#Directory> ;
    <http://www.w3.org/ns/ldp#contains> <Applications/>, <Family/>, <Friends/>, <Preferences/>, <Private/>, <Public/>, <Work/>, <profile/> ;
    <http://www.w3.org/ns/posix/stat#mtime> "1435333390" ;
    <http://www.w3.org/ns/posix/stat#size> "4096" .

<Applications/>
    a <http://www.w3.org/ns/ldp#BasicContainer>, <http://www.w3.org/ns/ldp#Container>, <http://www.w3.org/ns/posix/stat#Directory> ;
    <http://www.w3.org/ns/posix/stat#mtime> "1435333433" ;
    <http://www.w3.org/ns/posix/stat#size> "4096" .

However ldnode returns this:

@prefix n0: </>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix st: <http://www.w3.org/ns/posix/stat#>.
@prefix keys: </keys/>.
@prefix res: </resources/>.

<#>
    ldp:contains
        </acl.js>,
        </formats.js>,
        </http.js>,
        n0:keys,
        </params.js>,
        </patch.js>,
        n0:resources;
    st:mtime
       1437300761000;
    st:size
       306.
   </acl.js> st:mtime 1437139220000; st:size 42866 .
   </formats.js> st:mtime 1437090751000; st:size 1982 .
   </http.js> st:mtime 1437302617000; st:size 6750 .
   keys: st:mtime 1436972622000; st:size 340 .
   </params.js> st:mtime 1437089197000; st:size 3349 .
   </patch.js> st:mtime 1437090751000; st:size 5410 .
   res: st:mtime 1437302621000; st:size 986 .

Since warp looks for POSIX(Directory) in here https://github.com/linkeddata/warp/blob/gh-pages/src/app/list/list.js#L188

• Either we need to specify that attribute in ldnode
• We have to change how warp does that

Since package.json contains the script starts and the script test, is the Makefile still needed?
I am a bit confused of the references to tabulator and /dev/WWW (which I assume they were probably originally used for quick & dirty testing)

Update

What I am seeing now is that in order to run the tests one has to run ldnode with some specific settings and just by doing make you start ldnode ready for test settings. However this should all be done by npm test. The user doesn't want to see or do too many things when running the tests

Ldnode with multi-language support

• What is the purpose of this?
• Is it implemented?
• What is the current state?

(although other conversation have been raised, I open an issue just to keep track of this)

Also information on how to generate one should be provided

Assuming we want to start two ldnode running, editing the options' attribute will statically edit the options of each ldnode instance.

This problem does not arise in the current version of ldnode that has to be run from terminal, but it does in my fork where I am transforming ldnode into a library rather than a command line tool.

See my branch nicola/ldnode#routes for reference

I have seen in this file: https://github.com/linkeddata/ldnode/blob/master/handlers/get.js that the functions get, parseContainer, parseLinkedData, globHandler, containerHandler, fileHandler are declared as anonymous functions and assigned to a variable.

Since they are called before they assigned, this may lead to a TypeError or similar issues related to variable hoisting in javascript. http://jamesallardice.com/explaining-function-and-variable-hoisting-in-javascript/

So either we move up the variables, or we declare them, however I think declaring them is not a good idea either.

Instead of declaring them inside a function call (this means 1000 function calls 1000 declarations), we could instead just declare them outside and carry all the variables they need, I assume req, res being the only two. (This comment is valid for all the other files in /handlers that I have seen)

Along with authentication, RDF ACL authorization is an important feature for this server. Then we will be able to run it facing the open net rather than as a protected proxy.

The goal would be to make the file structure used on the server be compatible with gold, so that one could run node-ldp-httpd or gold on the same files and have redundancy of project, and no 'project lock-in'.

Also of course made sure node-rdf-acl works with any other express systems.

Turns out there's a bug in the webid module:

TypeError: Cannot call method 'replace' of undefined
  at new VerificationAgent (/devel/github.com/linkeddata/ldnode/node_modules/webid/index.js:21:23)
  at loginHandler (/devel/github.com/linkeddata/ldnode/login.js:20:30)
  at Layer.handle [as handle_request] (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/layer.js:95:5)
  at trim_prefix (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:312:13)
  at /devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:280:7
  at param (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:349:14)
  at param (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:365:14)
  at Function.process_params (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:410:3)
  at next (/devel/github.com/linkeddata/ldnode/node_modules/express/lib/router/index.js:271:10)
  at /devel/github.com/linkeddata/ldnode/index.js:125:24```