dat not sharing files other than dat.json about dat

[okdistribute]

sorry dat is currently unmaintained by a particular person but we're trying to fix that actively by spending some open collective money on a person
opencollective.com/dat

[RangerMauve]

We had a call today to talk about what would need to be done to get the CLI working again, specifically around dat-store.

The notes from the call can be found here: https://hackmd.io/XDxGGosQTvuCpeQwl5vahg?view

[corwin-of-amber]

I have the same problem with dat@latest on macOS.

[okdistribute]

Did you try the executable, available in the Releases section? I think the npm install -g dat is broken last I heard

[corwin-of-amber]

Yes, I noticed the other issues (e.g. #1186) and indeed the executable works, but I did want to try the new version...

[iankressin]

I'm facing the same issue on both of my machines after installing dat using npm i -g dat

Specs:
++++++++++++++++++

• Ubuntu 20.04
• Node version: 14.1.0
• Dat: 13.13.1

++++++++++++++++++

• Ubuntu 19.04
• Node version: 12.16.1
• Dat: 13.13.1

[iankressin]

@okdistribute I'm willing to help in anything you guys need. I just really like dat and I'll be glad to help

[RangerMauve]

@iankressin If you're interested in contributing time to get the Dat CLI working again, I'd be down to help coordinate that effort by giving pointers as to what needs to be done and reviewing PRs. It's a non-trivial amount of effort, however. 😅

[okdistribute]

@iankressin thanks ian! that would be super helpful. I think @RangerMauve is right, there is a non-trivial amount of 'potential' work, but I think getting it working with the latest Node is not that difficult, mostly updating dependencies and testing that it works. Since dat-node and dat are using package-lock.json, it falls behind easily when new Node versions come out. If you have time, we'd definitely merge a PR!

[iankressin]

@RangerMauve @okdistribute I'll definitely start taking a look at it in the next couple of days. @RangerMauve probably, I'll need a little bit of guidance at the beginning, so thanks for offering it.

[iankressin]

@RangerMauve I'm starting to take a look at the code and as the first action to try to solve this problem I upgraded the hypercore dependency from "hypercore": "^6.25" to "^9.5.7" (I know, it's a long way). The "empty folder" problem seems to be solved, but it won't pass 5 tests when running npm run test. All the errors have a identical message which is:

stderr was not empty ("(node:x) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency (Use node --trace-warnings ... to show where the warning was created) ")

The only difference between them being the node:x, where x varies in each error. The error message is not that helpful for debugging. I tried to search for padLevels in hypercore, tape, tape-spawn and in this project but found nothing.

What I'm looking for is better context for the error messages so I can debug faster. Am I in the right path?

Below, are all the 5 errors with a little bit of prior context:

# create - default opts no import                                                                                      
ok 76 creates dat directory
ok 77 matched pattern function                                                                                         
not ok 78 stderr was not empty ("(node:65189) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency (Use `node --trace-warnings ...` to show where the warning was created) ")
  ---                                                                                                                                                                                                                                          
    operator: ok
    expected: true
    actual:   false
    at: Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
    stack: |-
      Error: stderr was not empty ("(node:65189) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
      (Use `node --trace-warnings ...` to show where the warning was created)
      ")
          at Test.assert [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:260:54)
          at Test.bound [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Test.assert (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:379:10)
          at Test.bound [as ok] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
          at Socket.emit (events.js:327:22)
          at endReadableNT (_stream_readable.js:1223:12)
          at processTicksAndRejections (internal/process/task_queues.js:84:21)

# create - init alias
ok 86 creates dat directory
ok 87 matched pattern function
not ok 88 stderr was not empty ("(node:65223) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency (Use `node --trace-warnings ...` to show where the warning was created) ")
  ---
    operator: ok
    expected: true
    actual:   false
    at: Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
    stack: |-
      Error: stderr was not empty ("(node:65223) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
      (Use `node --trace-warnings ...` to show where the warning was created)
      ")
          at Test.assert [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:260:54)
          at Test.bound [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Test.assert (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:379:10)
          at Test.bound [as ok] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
          at Socket.emit (events.js:327:22)
          at endReadableNT (_stream_readable.js:1223:12)
          at processTicksAndRejections (internal/process/task_queues.js:84:21)

# create - with path
ok 90 creates dat directory
ok 91 matched pattern function
not ok 92 stderr was not empty ("(node:65234) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency (Use `node --trace-warnings ...` to show where the warning was created) ")
  ---
    operator: ok
    expected: true
    actual:   false
    at: Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
    stack: |-
      Error: stderr was not empty ("(node:65234) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
      (Use `node --trace-warnings ...` to show where the warning was created)
      ")
          at Test.assert [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:260:54)
          at Test.bound [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Test.assert (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:379:10)
          at Test.bound [as ok] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
          at Socket.emit (events.js:327:22)
          at endReadableNT (_stream_readable.js:1223:12)
          at processTicksAndRejections (internal/process/task_queues.js:84:21)

# keys - export & import secret key
ok 120 matched pattern function
ok 121 stderr was empty
ok 122 matched pattern function
not ok 123 stderr was not empty ("(node:65309) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency (Use `node --trace-warnings ...` to show where the warning was created) ")
  ---
    operator: ok
    expected: true
    actual:   false
    at: Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
    stack: |-
      Error: stderr was not empty ("(node:65309) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
      (Use `node --trace-warnings ...` to show where the warning was created)
      ")
          at Test.assert [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:260:54)
          at Test.bound [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Test.assert (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:379:10)
          at Test.bound [as ok] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
          at Socket.emit (events.js:327:22)
          at endReadableNT (_stream_readable.js:1223:12)
          at processTicksAndRejections (internal/process/task_queues.js:84:21)
  ...
ok 124 original dat file exists
ok 125 matched pattern function
not ok 126 stderr was not empty ("(node:65320) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency (Use `node --trace-warnings ...` to show where the warning was created) ")
  ---
    operator: ok
    expected: true
    actual:   false
    at: Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
    stack: |-
      Error: stderr was not empty ("(node:65320) Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency
      (Use `node --trace-warnings ...` to show where the warning was created)
      ")
          at Test.assert [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:260:54)
          at Test.bound [as _assert] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Test.assert (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:379:10)
          at Test.bound [as ok] (/home/kig/Documents/projects/dat/node_modules/tape/lib/test.js:84:32)
          at Socket.onEnd (/home/kig/Documents/projects/dat/node_modules/tape-spawn/index.js:106:14)
          at Socket.emit (events.js:327:22)
          at endReadableNT (_stream_readable.js:1223:12)
          at processTicksAndRejections (internal/process/task_queues.js:84:21)
  ...

[okdistribute]

@iankressin sorry I wasn't clear, some dependencies can't be updated without significant work, but others can -- hypercore & hyperdrive can't be updated without updating dat-storage which would require some deep understanding of hyperdrive 10

[iankressin]

@okdistribute No problem!! I'll get into hyperdrive as soon as possible.

Is there a specific part of the code where you suggest to begin with the refactor?

[iankressin]

@RangerMauve Got a better understanding on hyperdrive now. Are you still available to give some guidance on the refactoring?

[RangerMauve]

@iankressin Yeah sure, would you be down for a 1 hour call some time this week?

Tuesday is good for me between 11:00 ET and 22:00 ET, but I can do another day if you prefer. 😁

[iankressin]

@RangerMauve yepp, I'm down.

Tuesday is good for me. 20:00 ET??
Sent you a friend request in discord (found you at the hypercore channel)

[RangerMauve]

20:00 ET works for me. I'll follow up via DMs. 😁

[corwin-of-amber]

Thank you guys, this is wonderful news.