ipfs / benchmarks

Benchmarking for IPFS

IPFS is an open system to manage data without a central server

Check out our website at ipfs.tech.

For papers on IPFS, please see the Academic Papers section of the IPFS Docs.

License

MIT.

benchmarks's Issues

https://benchmarks.ipfs.team/ is now private?

Can we please open this up again?

Remote runner will fail for certain tests

The remote runner does not receive the complete JSON file if the test has more then 4 test files.

local-add.js] on host [147.75.33.155] for user [ubuntu] using [/home/ubuntu/.ssh/id_rsa]
INFO [1546118642520] (18857 on ip-172-31-44-15): running  [cat /tmp/out/unixFsAdd.json] on [147.75.33.155]
(node:18857) UnhandledPromiseRejectionWarning: SyntaxError: Unexpected end of JSON input

Add info in the README about repo layout, purpose, infra image, URLs for deployed services etc.

As discussed in #16.

benchmarks for go not working when using remote runner

INFO [1546178378903] (31093 on ip-172-31-44-15): running  [cat /tmp/out/unixFsAddGo.json] on [147.75.33.155]
(node:31093) UnhandledPromiseRejectionWarning: Error: (node:29024) UnhandledPromiseRejectionWarning: Error: spawn ipfs ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:232:19)
    at onErrorNT (internal/child_process.js:407:16)
    at process._tickCallback (internal/process/next_tick.js:63:19)
(node:29024) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:29024) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

    at remoteExec (/data/ipfs-benchmarks/runner/remote.js:22:18)
    at f (/data/ipfs-benchmarks/runner/node_modules/once/once.js:25:25)
    at Duplexify.<anonymous> (/data/ipfs-benchmarks/runner/node_modules/ssh-exec-plus/index.js:155:5)
    at Duplexify.emit (events.js:187:15)
    at endReadableNT (/data/ipfs-benchmarks/runner/node_modules/duplexify/node_modules/readable-stream/lib/_stream_readable.js:1010:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
(node:31093) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:31093) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Is IPFS installed on the benchmark server?

Verify warmup is working as expected

Alerts are need if remote runner job fails on nightly cron job ( or doesn't run )

We need an alert either on a failed job or missing the daily points in a graph using Grafana ( not sure if Grafana has this feature)

Remote runner with clinic hangs

Add memory consumption to grafana

As titled

IPFS can't find lock file

version: 0.34.0-pre.0

This is happening on the 5 node tests on master branch only.

    Error: /home/ubuntu/ipfs/js-ipfs/node_modules/proper-lockfile/lib/lockfile.js:181
            onCompromised: (err) => { throw err; },
                                      ^

    Error: ENOENT: no such file or directory, utime 'data/ipfs/853976/repo.lock'

Format the uploaded result.json files so that they are human readable

JSON.stringify(data, null, 2).

Create tests

• Node initialization - Node.js (timing around creating a repo without pregen key)
• Node initialization - Browser
• Add files - Browser
• Cat files (websocket, mplex, secio) js0 -> js1 - Browser
• Cat files (websocket, mplex) - Node.js
• Cat files (websocket, mplex, secio) js01234->js5 - Node.js

Up date tests doc

@litzenberger https://github.com/ipfs/benchmarks/blob/master/tests/README.md has no mention of the multi-peer-transfer test

I think it generally could use more content

• how the tests work
• explain architecture

Catch up summary 2018-11-21

Apologies I wanted to get this done yesterday. Hopefully this captures our conversation:

Summary

@Elexy gave a presentation demoing the current setup. 3 basic benchmark tests currently.

Infrastructure requirements were ironed out with @eefahy. See conversation here ipfs/infra#452 (comment) for details.

@mcollina identified a perf issue for node initialization and documented here ipfs/js-ipfs#1727

Agreed a weekly catch up at the same time (Wednesday 15:00).

Action items

• Update infra diagram (https://github.com/ipfs/benchmarks/blob/master/docs/images/prod-infrastructure.png) (@Elexy) - #15
• @Elexy to provide @eefahy with public key for access to runner vm
• Infrastructure is set up (@eefahy)
• README to be updated with repo layout, purpose, infra image, URLs for deployed services etc. (@litzenberger @Elexy)
• Add instructions for how to run locally (@litzenberger)
• Add instructions for how to add benchmarks (@litzenberger) #17
• Instruct Nearform on pregenerated key usage (@alanshaw)
• Investigate potential issue running benchmarks with existing repo (@alanshaw)
• Run locally and feedback (@alanshaw, @achingbrain, @hugomrdias)

automate deployment of grafana dashboard

How long do you think that would take @Elexy?

Add a page in our benchmarks about all the runs with the links to ifs

Basically we should:

• upload all the result and clinic data into a folder, adding it to ipfs will return an hash
• store the hash for each run that we do in influxdb or a JSON file
• add an html page in our runner that gets that data, and display them in a list. It would be fantastic to also include a number from one of the benchmarks, and maybe a quick summary of all the data for each run.
• add pagination to that list

---

@litzenberger can you do the necessary work in the runner to add this feature?

@Elexy can you sort out the uploading to ipfs?

benchmark site unavailable

https://benchmarks.ipfs.team/

This site can’t be reached benchmarks.ipfs.team refused to connect.
Try:

Checking the connection
Checking the proxy and the firewall
ERR_CONNECTION_REFUSED

nginx docker is not starting - log file ->

2018/12/30 04:12:22 [emerg] 1#1: open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/default.conf:22
nginx: [emerg] open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/default.conf:22
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback```

libp2p node is not started yet

$ STAGE=remote node multi-peer-transfer.js
Verifying File OneKBFile
Verifying Directory Hundred1KBFile
Verifying File One62KBFile
Verifying File One64KBFile
Verifying File One512KBFile
Verifying File One768KBFile
Verifying File One1023MBFile
Verifying File OneMBFile
Verifying File One4MBFile
Verifying File One8MBFile
Verifying File One64MBFile
Verifying File One128MBFile
Verifying File One512MBFile
Verifying File OneGBFile
Files Verified
Swarm listening on /ip4/127.0.0.1/tcp/4012/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/192.168.1.222/tcp/4012/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/127.0.0.1/tcp/4022/ws/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/127.0.0.1/tcp/4013/ipfs/QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
Swarm listening on /ip4/192.168.1.222/tcp/4013/ipfs/QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
Swarm listening on /ip4/127.0.0.1/tcp/4023/ws/ipfs/QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
Swarm listening on /ip4/127.0.0.1/tcp/4014/ipfs/QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
Swarm listening on /ip4/192.168.1.222/tcp/4014/ipfs/QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
Swarm listening on /ip4/127.0.0.1/tcp/4024/ws/ipfs/QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
Swarm listening on /ip4/127.0.0.1/tcp/4015/ipfs/QmeKizHABFpHtEZoJiuZJmvVtC1yXi2MrRZ3UqgnqrUqet
Swarm listening on /ip4/192.168.1.222/tcp/4015/ipfs/QmeKizHABFpHtEZoJiuZJmvVtC1yXi2MrRZ3UqgnqrUqet
Swarm listening on /ip4/127.0.0.1/tcp/4025/ws/ipfs/QmeKizHABFpHtEZoJiuZJmvVtC1yXi2MrRZ3UqgnqrUqet
Swarm listening on /ip4/127.0.0.1/tcp/4016/ipfs/QmdX7uA1nQq1oghmxHhpKfb7VbvjETPd6DihW4w5Ht52Tt
Swarm listening on /ip4/192.168.1.222/tcp/4016/ipfs/QmdX7uA1nQq1oghmxHhpKfb7VbvjETPd6DihW4w5Ht52Tt
Swarm listening on /ip4/127.0.0.1/tcp/4026/ws/ipfs/QmdX7uA1nQq1oghmxHhpKfb7VbvjETPd6DihW4w5Ht52Tt
Writing output in a multiple files
┌────────────────────┬────────────────────┬──────────────────────────────────────────────────┬────────────────────┬────────────────────┐
│ Test               │ Sub Test           │ Description                                      │ File Set           │ Duration           │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ empty-repo         │ Description of benchamrk                         │ OneKBFile          │ s:0 ms: 364.174486 │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ empty-repo         │ Description of benchamrk                         │ OneMBFile          │ s:0 ms: 734.050394 │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ empty-repo         │ Description of benchamrk                         │ One4MBFile         │ s:0 ms: 945.971094 │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ empty-repo         │ Description of benchamrk                         │ One64MBFile        │ s:6 ms: 503.684288 │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ populated-repo     │ Description of benchamrk                         │ OneKBFile          │ s:0 ms: 1.532656   │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ populated-repo     │ Description of benchamrk                         │ OneMBFile          │ s:0 ms: 4.923811   │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ populated-repo     │ Description of benchamrk                         │ One4MBFile         │ s:0 ms: 8.442353   │
├────────────────────┼────────────────────┼──────────────────────────────────────────────────┼────────────────────┼────────────────────┤
│ multiPeerTransfer  │ populated-repo     │ Description of benchamrk                         │ One64MBFile        │ s:0 ms: 273.107513 │
└────────────────────┴────────────────────┴──────────────────────────────────────────────────┴────────────────────┴────────────────────┘
assert.js:350
    throw err;
    ^

AssertionError [ERR_ASSERTION]: The libp2p node is not started yet
    at Node.dialProtocol (/Users/matteo/Repositories/js-ipfs/node_modules/libp2p/src/index.js:236:5)
    at Network._dialPeer (/Users/matteo/Repositories/js-ipfs/node_modules/ipfs-bitswap/src/network.js:165:17)
    at Network.sendMessage (/Users/matteo/Repositories/js-ipfs/node_modules/ipfs-bitswap/src/network.js:129:10)
    at DecisionEngine._sendSafeBlocks (/Users/matteo/Repositories/js-ipfs/node_modules/ipfs-bitswap/src/decision-engine/index.js:83:18)
    at eachSeries (/Users/matteo/Repositories/js-ipfs/node_modules/ipfs-bitswap/src/decision-engine/index.js:65:14)
    at /Users/matteo/Repositories/js-ipfs/node_modules/async/internal/withoutIndex.js:9:16
    at replenish (/Users/matteo/Repositories/js-ipfs/node_modules/async/internal/eachOfLimit.js:66:17)
    at iterateeCallback (/Users/matteo/Repositories/js-ipfs/node_modules/async/internal/eachOfLimit.js:50:17)
    at /Users/matteo/Repositories/js-ipfs/node_modules/async/internal/onlyOnce.js:12:16
    at _sendSafeBlocks (/Users/matteo/Repositories/js-ipfs/node_modules/ipfs-bitswap/src/decision-engine/index.js:71:11)

Link to grafana in the README

As titled, it would be very useful

no results for FILESET=One512MBFile

$ STAGE=local REMOTE=true FILESET=One512MBFile VERIFYOFF=true node local-transfer.js
set up observer
new stat object 1
Swarm listening on /ip4/127.0.0.1/tcp/4012/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/192.168.1.222/tcp/4012/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/127.0.0.1/tcp/4022/ws/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
set up observer
new stat object 2
Swarm listening on /ip4/127.0.0.1/tcp/4013/ipfs/QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
Swarm listening on /ip4/192.168.1.222/tcp/4013/ipfs/QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
Swarm listening on /ip4/127.0.0.1/tcp/4023/ws/ipfs/QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
new peer QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
new stat object 3
new stat object 4
new stat object 5
new peer QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
new stat object 6
new stat object 7
new stat object 8
new stat object 9
new stat object 10
new stat object 11
new stat object 12
set up observer
new stat object 13
Swarm listening on /ip4/127.0.0.1/tcp/4014/ipfs/QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
Swarm listening on /ip4/192.168.1.222/tcp/4014/ipfs/QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
Swarm listening on /ip4/127.0.0.1/tcp/4024/ws/ipfs/QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
new peer QmNc18FMrKks42Nd2PVwtBHdSvmfkhcvANDBe9byVuXd3f
new stat object 14
new stat object 15
new stat object 16
new peer QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
new stat object 17
new peer QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
new stat object 18
new peer QmbZRmAfarFJYDUwa5WSQ1FazCQ72iA7Hp3qE2ZPhRF7is
new stat object 19
new stat object 20
new stat object 21
Writing output in a multiple files
┌────────────────────┬────────────────────┬──────────────────────────────────────────────────┬────────────────────┬──────────┐
│ Test               │ Warmup             │ Description                                      │ File Set           │ Duration │
└────────────────────┴────────────────────┴──────────────────────────────────────────────────┴────────────────────┴──────────┘
Removed ./data/ipfs/

Catch up summary 2019-01-02

https://cryptpad.fr/code/#/2/code/edit/x1BPpbuVDcAg4yjMlmFpx1x6/

Action points

• Arrange walk through for libp2p-switch and mplex, multistream-select - @alanshaw (Thursday morning or Friday ideally)
• CI trigger commit SHA must be propagated - @Elexy
• Video recording of runner and dashboard - @Elexy (#140)
• Write down requirements for trigger from CI (use Gitlab, no need for infra involvement) - @alanshaw
• Review ipfs/js-datastore-fs#21 - @alanshaw / @achingbrain

Running dashboard locally

./scripts/runLocalEnv.sh up

Does not work for me.

Catch up summary 2018-11-28

https://cryptpad.fr/code/#/2/code/edit/JWof3cTi1gXKSc+W-x6zPwXg/

Action items

• Clear documentation exists for provisioning the machines locally (@Elexy)
• Complete documentation for adding benchmark tests (@litzenberger)
• Complete and document trigger HTTP endpoint (@Elexy)
• Deploy to production machines (@litzenberger / @Elexy)
• Ensure dashboard is available at benchmarks.cloud.ipfs.team (@litzenberger / @Elexy)
• Document creating Peer ID (for pregenerated private key) in js-ipfs README (@alanshaw) ipfs/js-ipfs#1770
• Create matrix of additional benchmark tests we'd like (@alanshaw) https://gist.github.com/alanshaw/e8f49ff1875f0e256e7c45ddeef31498
• Initiate conversation with cluster team for storing artefacts in IPFS (@alanshaw)

sizes of files to add

We are currently uploading a 1 MB and 10KB files. What are representatives numbers for ipfs? Should we benchmark with 10 MB, 100 MB and maybe even 1 GB?

@alanshaw can you help with some guidance here?

Deploy dashboard documentation

Store benchmarks results in IPFS

@alanshaw Can you offer any guidance on how we can store the results of the benchmarks on IPFS?
Any specific location or format?

document how to test against master

This seems very relevant because of ipld/js-ipld-dag-pb#109

multi-peer-transfer.js not exiting 100% of times

Some times the benchmark stays running, it seems we are not closing all nodes properly.

Do a video recording of the Runner/Architecture and Dashboard

As titled and discussed in #44.

verify that discovery for new peers is disabled

as titled.

How do we do that @alanshaw @daviddias?

Verify master branch js-ipfs is being used on remote tests

The current version of js-ipfs when running remote tests is version 0.33.1 is but the master branch version is is '0.34.0-pre.0'

Catch up summary 2018-12-05

https://cryptpad.fr/code/#/2/code/edit/QoUwIsoJCT+8mfK1TZTMuIo3/

Action items

• Video with demo of dashboard (@Elexy) - #44
• Integrate trigger with CI (@Elexy) - ipfs/infra#464
• Setup storing things on IPFS (@Elexy)
• Setup and demo clinic (demo scheduled for dec 11) (@mcollina)
• Browser tests ongoing (@litzenberger)
• Review and prioritise https://gist.github.com/alanshaw/e8f49ff1875f0e256e7c45ddeef31498 (@mcollina @alanshaw)

Change default time visualized in Grafana to the last 30 days

as titled

Fix browser warmup value in results

Need to change warmup value from true to off to be consistent with the other tests.

No such file or directory: './infrastructure/local/docker-compose.yaml'

I'm running:

docker-compose -f infrastructure/local/docker-compose.yaml up

From https://github.com/ipfs/benchmarks/blob/master/runner/README.md

catch up summary 2018-12-12

https://cryptpad.fr/code/#/2/code/edit/Aj4GGxNX7sqcf+NCX5lzlDp1/

No action points

Error when running locally with Dashboard

rons-MBP:benchmarks ronlitzenberger$ STAGE=local LOG_PRETTY=true node runner/cli.js
INFO [1544553155694] (99751 on rons-MBP.attlocal.net): Running [OUT_FOLDER=/tmp/out  node /Users/ronlitzenberger/nearform/ipfs/benchmarks/tests/local-transfer.js] locally
INFO [1544553298798] (99751 on rons-MBP.attlocal.net): Retrieving [cat /tmp/out/localTransfer.json] locally
INFO [1544553298807] (99751 on rons-MBP.attlocal.net):
    err: {
      "type": "Error",
      "message": "Command failed: cat /tmp/out/localTransfer.json\ncat: /tmp/out/localTransfer.json: No such file or directory\n",
      "stack":
          Error: Command failed: cat /tmp/out/localTransfer.json
          cat: /tmp/out/localTransfer.json: No such file or directory

              at ChildProcess.exithandler (child_process.js:289:12)
              at ChildProcess.emit (events.js:182:13)
              at maybeClose (internal/child_process.js:962:16)
              at Process.ChildProcess._handle.onexit (internal/child_process.js:251:5)
      "killed": false,
      "code": 1,
      "signal": null,
      "cmd": "cat /tmp/out/localTransfer.json"
    }
    stdout: ""
    stderr: "cat: /tmp/out/localTransfer.json: No such file or directory\n"
ERROR [1544553298808] (99751 on rons-MBP.attlocal.net): cat: /tmp/out/localTransfer.json: No such file or directory

    Error: cat: /tmp/out/localTransfer.json: No such file or directory

        at exec (/Users/ronlitzenberger/nearform/ipfs/benchmarks/runner/local.js:31:20)
        at ChildProcess.exithandler (child_process.js:296:5)
        at ChildProcess.emit (events.js:182:13)
        at maybeClose (internal/child_process.js:962:16)
        at Process.ChildProcess._handle.onexit (internal/child_process.js:251:5

subTest.warmup.toLowerCase is not a function

$ STAGE=local REMOTE=true VERIFYOFF=true node init-node.js
Swarm listening on /ip4/127.0.0.1/tcp/4012/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/192.168.1.222/tcp/4012/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
Swarm listening on /ip4/127.0.0.1/tcp/4022/ws/ipfs/QmSxYJBid9EUdV61T9dUELAsc2FncyQWaSEvm4ztcURiur
subTest.warmup.toLowerCase is not a function

IPFS not installed on benchmark

Benchmark playbook error:

structure/playbooks/benchmarks.yaml] locally
ERROR [1546307012481] (26103 on ip-172-31-44-15): Error:  [WARNING]: Consider using the get_url or uri module rather than running wget.
If you need to use command because get_url or uri is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to
get rid of this message.

    Error: Error:  [WARNING]: Consider using the get_url or uri module rather than running wget.
    If you need to use command because get_url or uri is insufficient you can add
    warn=False to this command task or set command_warnings=False in ansible.cfg to
    get rid of this message.

        at Object.ensure (/data/ipfs-benchmarks/runner/provision.js:72:11)
        at process._tickCallback (internal/process/next_tick.js:68:7)

missing LICENSE file

Which LICENSE file should we include in this project?

Browser tests - Need libraries installed on benchmark server

Browser tests - we get the error /home/ubuntu/ipfs/tests/node_modules/puppeteer/.local-chromium/linux-609904/chrome-linux/chrome: error while loading shared libraries: libX11-xcb.so.1: cannot open shared object file: No such file or directory

I haven't tested this on the server but according to doc we need this ran on the benchmark server
sudo apt-get install libpangocairo-1.0-0 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxi6 libxtst6 libnss3 libcups2 libxss1 libxrandr2 libgconf2-4 libasound2 libatk1.0-0 libgtk-3-0

Verify test files are created before running tests

Verify test files are created before running tests

Broken on Node 11

$ nvm use 11
node Now using node v11.3.0 (npm v6.4.1)
$ node local-add.js
(node:94288) UnhandledPromiseRejectionWarning: Error: Callback was already called.
    at /Users/matteo/Repositories/ipfs-benchmarks/tests/node_modules/async/internal/onlyOnce.js:9:32
    at _locker.lock (/Users/matteo/Repositories/ipfs-benchmarks/tests/node_modules/ipfs-repo/src/index.js:170:16)
    at lock.then.catch.err (/Users/matteo/Repositories/ipfs-benchmarks/tests/node_modules/ipfs-repo/src/lock.js:40:19)
(node:94288) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:94288) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Maybe it's not us, we might need to escalate.

GO test fails on benchmark

INFO [1546370824144] (32112 on ip-172-31-44-15): running  [cat /tmp/out/unixFsAddGo.json] on [147.75.33.155]
ERROR [1546370824144] (32112 on ip-172-31-44-15): Daemon exited with code: 0

    Error: Daemon exited with code: 0

        at remoteExec (/data/ipfs-benchmarks/runner/remote.js:22:18)
        at f (/data/ipfs-benchmarks/runner/node_modules/once/once.js:25:25)
        at Duplexify.<anonymous> (/data/ipfs-benchmarks/runner/node_modules/ssh-exec-plus/index.js:155:5)
        at Duplexify.emit (events.js:187:15)
        at endReadableNT (/data/ipfs-benchmarks/runner/node_modules/duplexify/node_modules/readable-stream/lib/_stream_readable.js:1010:12)
        at process._tickCallback (internal/process/next_tick.js:63:19)
INFO [1546370824145] (32112 on ip-172-31-44-15): not running doctor: off

Move tests/local-extract.js to just use cat

Add tests to runner/dashboard

There are several tests that need to be added to remote runner and dashboard.

• Initial browser test
• Small 100 1 KB multi file test
• Initialize node
• Local add For go

Unable to store some tests in DB

(node:21402) UnhandledPromiseRejectionWarning: Error: A 400 Bad Request error occurred: {"error":"unable to parse 'multiPeerTransfer,warmup=off,commit=7bcc49615e54f09c34bae07c994e3be3d870fe1f,project=js-ipfs,file_set=OneKBFile,version=0.34.0-rc.0,repo=7,guid=33c04690-0d29-11e9-95c6-a387bf343229,sha=tbd,branch=master duration=468.717883,ipfs_sha=undefined 1546281070266000000': invalid boolean\nunable toparse 'multiPeerTransfer,warmup=off,commit=7bcc49615e54f09c34bae07c994e3be3d870fe1f,project=js-ipfs,file_set=OneMBFile,version=0.34.0-rc.0,repo=7,guid=33c04690-0d29-11e9-95c6-a387bf343229,sha=tbd,branch=master duration=974.796503,ipfs_sha=undefined 1546281070266000000': invalid boolean\nunable to parse 'multiPeerTransfer,warmup=off,commit=7bcc49615e54f09c34bae07c994e3be3d870fe1f,project=js-ipfs,file_set=One4MBFile,version=0.34.0-rc.0,repo=7,guid=33c04690-0d29-11e9-95c6-a387bf343229,sha=tbd,branch=master duration=1434.165117,ipfs_sha=undefined 1546281070266000000': invalid boolean\nunable to parse 'multiPeerTransfer,warmup=off,commit=7bcc49615e54f09c34bae07c994e3be3d870fe1f,project=js-ipfs,file_set=One64MBFile,version=0.34.0-rc.0,repo=7,guid=33c04690-0d29-11e9-95c6-a387bf343229,sha=tbd,branch=master duration=8021.973649,ipfs_sha=undefined 1546281070266000000': invalid boolean"}

    at new RequestError (/data/ipfs-benchmarks/runner/node_modules/influx/lib/src/pool.js:52:28)
    at IncomingMessage.<anonymous> (/data/ipfs-benchmarks/runner/node_modules/influx/lib/src/pool.js:62:53)
    at IncomingMessage.emit (events.js:187:15)
    at endReadableNT (_stream_readable.js:1094:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
(node:21402) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 5)
(node:21402) UnhandledPromiseRejectionWarning: Error: A 400 Bad Request error occurred: {"error":"unable to parse 'addMultiKb,warmup=off,commit=7bcc49615e54f09c34bae07c994e3be3d870fe1f,project=js-ipfs,file_set=Hundred1KBFile,version=0.34.0-rc.0,repo=7,guid=33c04690-0d29-11e9-95c6-a387bf343229,sha=tbd,branch=master duration=4535.500524,ipfs_sha=undefined 1546281070267000000': invalid boolean"}

    at new RequestError (/data/ipfs-benchmarks/runner/node_modules/influx/lib/src/pool.js:52:28)
    at IncomingMessage.<anonymous> (/data/ipfs-benchmarks/runner/node_modules/influx/lib/src/pool.js:62:53)
    at IncomingMessage.emit (events.js:187:15)
    at endReadableNT (_stream_readable.js:1094:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
(node:21402) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 6)

use catReadableStream instead of cat

Create a guide on how to add new benchmarks

As discussed in #16.

Fix 504 Gateway on API

As titled

catch up summary 2018-12-19

https://cryptpad.fr/code/#/2/code/edit/TcVY3ac2pvYx5tTihdCYd2MT/

Action points

• @mcollina to ask Go IPFS team to point to their implementaiton of atomic file writing ipfs/kubo#5863
• Chunks are written serially - is that a bi-product of using pull streams or is there some other reason? @achingbrain ipfs-inactive/js-ipfs-unixfs-importer#9
• @litzenberger to assign browser tests PR to @alanshaw to help point in the right direction