nodefactoryio / hactar-backend

Node.js backend application for Hactar (Filecoin tool for mining analysis)

Hactar backend

Node.js application for Hactar API.

This repository contains API implementation for Hactar, a Filecoin mining analyzer.

Development

Requirements

Following software is required to be installed to use this repo:

• NodeJs >= v8.4.0
• Yarn
• Docker
• docker-compose

Usage

Make sure to run yarn install before first build or each time you change dependencies in package.json.

On first use of this repo, run npx task build which will build docker image.You will have to run npx task build each time you change dependencies in package.json (yarn.lock).

Run npx task --help to see all available commands and their description.

Notice

• make sure you update yarn.lock before building
• use sequelize-cli local to generate migrations (because of timestamp)

Documentation

The documentation can be accessed on app-url:${SWAGGER_DOCS_PORT}/${SWAGGER_DOCS_URL}

e.g. http://localhost:8080/api/swagger/

• default docs url: /api/swagger
• default docs port: 8080

License

This project is dual-licensed under Apache 2.0 and MIT terms:

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

hactar-backend's Issues

Fetch mining rewards history by filters

Expose nodes where user can fetch (using node ID) mining rewards history filtering them by:

• week
• day
• month
• year

Allow this filter to be query params i.e. ?filter=week

Fix compiling migrations

Fix auth response

Response from api/user/login should be json response and not plain string. I propose most simple structure as {token: "dummy-token-123"}

Read sectors

Create a service that will read used sectors - https://docs.google.com/document/d/1LebOq0QCI_P8AZPwzyB12vbtWXVfB2VXnJRUV-TDJRg/edit#heading=h.cq6o5h7w3b

Expose API endpoint for frontend that will pass result from calling this service. Note, we don't store anything to db here, we serve it just as a proxy because we have the token permission.

---

TBD

Store payment channels

Create a route that will store latest payment channels from a node.

API endpoint receives the node URL and address and list of payment channels with the following information:

• payment channel status
• voucher list with amount, signature and timelock

If payment channel for that node already exists in db, replace it with new information.

Seeders

Create seeders / dummy data for models we currently have

Edit account

Create endpoint PUT user/account to allow user to update account email or password.

Storing miner nodes

User can have N numbers of miner nodes (create new database model).
Expose endpoint that will allow user to store miner node with the following data:

• URL
• miner token (provided by user, not our JWT)
• actor address

Expose delete endpoint too.

Read power

Create a service that will read miner power - https://docs.google.com/document/d/1LebOq0QCI_P8AZPwzyB12vbtWXVfB2VXnJRUV-TDJRg/edit#heading=h.z0rdn7wdf3xs

Expose API endpoint for frontend that will pass result from calling this service. Note, we don't store anything to db here, we serve it just as a proxy because we have the token permission.

---

TBD

Support adding node name and description

Allow user to append node name and description to an existing node.

Create:

• new columns migration for this (adding name and description in Node model)
• route for updating this node info

Expose route for fetching node status

Need route for fetching node status that includes:

• Node latest uptime record
• Node latest disk space record

Store mining rewards

Create an endpoint that will allow storing a log for miner rewards (receive array). This endpoint will be called when node detects that a miner has received a reward.

Reward amount is a fixed constant and saved in our backend app.

Should store owner address, miner actor, amount and time.

Endpoint API should also receive node URL and address.

Example of input:

[
   {
     Cid:   "bafy2bzaceawp7zcx74biecfl3axvoulh4lgdnnwzvgaza2cdhmxx75ymo6jnk",
     Address: "t0101",
     URL: "..."
   },
]

Amount is a fixed constant and should be in env/config file.

Fetch uptime history by filters

Expose nodes where user can fetch (using node ID) uptime records history filtering them by:

• week
• day
• month
• year

Allow this filter to be query params i.e. ?filter=week

Logger not working with unit tests

Expected: Have info logs in tests

Long lived token

Create JWT separately for daemon app that is long lived (no expiration).

Should be on separate route than current login.

Store pledged collateral

TBD

Store disk space

Expose endpoint that will allow storing disk space which includes the following data:

• total free space (in bytes, numeric)
• taken space (in bytes, numeric)

Endpoint API should also receive node URL and actor address (we use both to distinguish unique nodes).

Create model schema accordingly, I propose naming DiskInformation. Note that this model depends (has relationship where 1 Node can have N disk information) on the Node model :)

Offline node notification

When offline node is detected, send user email notification.

Refactor past deals endpoint

Registration

Implement registration using email and password.

1. Create user model with email, hash_password fields
2. Create auth/register endpoint
3. Return created user on successful registration.

Note, we will add later email confirmation. We will use JWT for authentication.

Docs for endpoints

Use Swagger

Login

Authentication

User should be able to register using email and password. We use JWT for authentication and authorization.

Todo:

• create /user/register endpoint
• create /user/login endpoint

Email confirmation or forgotten password or update password is NOT required as a part of this ticket.

Fix CI for production build

Build production image steps fails in Circle CI

Step 1/9 : FROM nodefactory/nodejs-ts-starter:latest
pull access denied for nodefactory/nodejs-ts-starter, repository does not exist or may require 'docker login'

Store past deals

Create a service that will store past deals from node - https://docs.google.com/document/d/1LebOq0QCI_P8AZPwzyB12vbtWXVfB2VXnJRUV-TDJRg/edit#heading=h.8zo955pdphcy

API endpoint receives the node URL, address and deal info: CID, state, size, provider, price, duration.

We will not store this by time and therefore should always have 1 entry per node (use PUT).

Enable pagination for deals

Allow fetching of past deals with pagination.

Use query params (from, to) for pages.

Email service

Create an email service that will be used for sending email.

EDIT: use sendinblue

Try to make it testable

Fix e2e tests in CI

Try to fix e2e tests in CircleCI by running migrations first, before test.

Read sector size

Create a service that will read defined sector size - https://docs.google.com/document/d/1LebOq0QCI_P8AZPwzyB12vbtWXVfB2VXnJRUV-TDJRg/edit#heading=h.17gaedko9u8a

Expose API endpoint for frontend that will pass result from calling this service. Note, we don't store anything to db here, we serve it just as a proxy because we have the token permission.

---

TBD

Store uptime

Expose simple endpoint (and create schema model) that will store reported uptime. This endpoint will be used to report if node is working or not so values can be true and false. Also, need time when entry was created.

Endpoint API should also receive node URL and actor address.

Read client version

Create a service that will read node client's version - https://docs.google.com/document/d/1LebOq0QCI_P8AZPwzyB12vbtWXVfB2VXnJRUV-TDJRg/edit#heading=h.k29oq15rijqx

Expose API endpoint for frontend that will pass result from calling this service. Note, we don't store anything to db here, we serve it just as a proxy because we have the token permission.

API endpoint receives only the node URL.

Authorization

Write middleware that will check if user's JWT is valid + user is allowed to access a specific records (is user record owner). Use it on existing routes:

• creating and deleting user node
• creating disk info
• creating and fetching node uptime

or any other that I might have missed or been added meanwhile :)

Reset account password

Use email to reset account password (forgotten password).

Validation

Enable validation middleware (need to integrate with typescript)

Add number of sectors

Expand general / miner info with number of sectors

Disabling notifications

Save in database if user has enabled or disabled email notifications for node.

Route for fetching disk info

Create a route that user will use to fetch disk info for a specific node (/node/diskinfo/:id).
Use JWT token for authorization, throw Forbidden if user not authorized.

Fetch balance

Create route for fetching balance using node ID.

We should display user's current balance (last entry) and 24 hour change (i.e. +149.FIL (0.5%)) -> TBD how

Store balance

Expose endpoint that will allow storing user's balance by time.

Create new model schema that stores balance number and time when entry is created.

Note that balance number is saved as bigint as we will be working with big numbers i.e. 139394994494949 (e^18)

Endpoint API should receive node URL, address and user's balance.

Enable Circle CI

Fetch disk space history by filters

Expose nodes where user can fetch (using node ID) disk space records history filtering them by:

• week
• day
• month
• year

Allow this filter to be query params i.e. ?filter=week

Route for fetching user nodes

Create a route that will fetch all nodes by a user. Use JWT token.

Fix filters

We should not return ALL records but only the latest one appropriate for that filter.
For example, if filter week is used we should only return 7 records

Registration confirmation via email

Use email to confirm and complete account registration.
Generate unique confirmation ID that doesn't expire.

Return node status in period by filter

TBD

Read user balance

Create a service that will read balance from node's address - https://docs.google.com/document/d/1LebOq0QCI_P8AZPwzyB12vbtWXVfB2VXnJRUV-TDJRg/edit#heading=h.ake6zkp56144

Expose API endpoint for frontend that will pass result from calling this service. Note, we don't store anything to db here, we serve it just as a proxy because we have the token permission.

---

TBD

Fix migrations

Store general miner info

Store type of information that is regularly updated but doesn't need to have history for graphs or similar like we did with other information like disk information.

Create new table and endpoint that receives:

• version (string i.e. 0.11.0)
• sector size (bytes, bigint)
• minerpower (bytes, bigint)
• totalpower (bytes, bigint)

Node URL and address is received together with the info.

Note that we use only PUT type of requests here.
Also create a GET route so frontend can fetch this by node ID.

Response when no node balance

Return current Balance: 0 when there is no updated balance in the last 24 hours.

Expand general miner info with wallet address