arkecosystemarchive / ark-node Goto Github PK

Currently all delegates have to setup their individual payment scripts. It would be really cool if we can save the delegates that trouble and ship a good configurable payment script with the node.

when we select "S" for Node Status and we not input write address it will ask again and again for the address and there is no way to exit from the loop! "Enter delegate ADDRESS, NOT the SECRET"

It seems that new nodes are unable to initially sync with the testnet.

This can be reproduced by starting a new node from scratch and installing the build script from https://ark.io/installARK.sh. This doesn't seem to affect nodes which were already syncing 3 days ago.

When starting the node, an UnhandledPromiseRejectionWarning exception is thrown, and the node hangs indefinitely.

Some simple debugging of app.js shows that the exception is thrown while requiring ark.js in the logic/blocks.js file.

I wonder if this is a problem when importing the genesis block, or if the change to addressing has caused nodes without any blocks to stop importing blocks returned from the network.

I did set my delegate correctly and it is forging. But the Status window does not show it.

--

I used mainnet and in logs I had:

[inf] 2018-10-12 10:10:25 | http://5.39.53.54:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://5.39.53.55:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.160:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.161:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.162:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.163:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.164:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.165:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.166:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.167:4001 banned for 360 minutes
[inf] 2018-10-12 10:10:25 | http://37.59.129.168:4001 banned for 360 minutes
[inf] 2018-10-12 10:11:25 | http://5.39.53.51:4001 banned for 360 minutes
...

because of that I could not receive incoming payments and my node was out of sync for long time. It happened right after I updated my node till 1.3.1 on this Saturday.

Hi,

I believe you have a typo in the "ARKcommander.sh" script. The URL "https://texplorer.ark.io/current" should be "https://explorer.ark.io/current".

List transactions endpoint returns Transaction blockid field (all lowercase). It should be returned as blockId (camel-case) to be consistent with other properties.

curl localhost:4000/api/transactions?limit=1

{
  "success":true,
  "transactions":[
  {
	"id": "fbb92bb06261805f37a513e36e43a7256ef6850a69f2cf8574f609ee6fbc303e",
	"blockid": "17504930779201596224",
	"type": 0,
	"timestamp": 0,
	"amount": 0,
	"fee": 0,
	"senderId": "aAFCkU3SV1TzNfjgxdewVH3LvCwvY92mCM",
	"recipientId": "a3wN4KwRKS2vLu2GWqMHXNoC2iryR6qdVn",
	"senderPublicKey": "0240143aa7eddb726aae5faa588d943f6ff99976afecefc13563ebce3aaec32d56",
	"signature": "3045022100b79240b612c5bad01f93bc4116d72a03faba402c2b7f884455d9e5413e99923602205e40965b57c904b7adeaf6fbeda13d7c42442443c333269f4a275c8ac5c0d6fb",
	"asset": {},
	"confirmations": 1004
}
]

Making this change might break existing clients.

We should add system tests that check API response structure to ensure we don't make "breaking" changes on the API contract.

Breaking API changes might include:

• renaming a property
• removing a property
• changing the type of a property
• moving the property to a different nesting level
• making a required property optional

We should add this in a way that makes it easy to add the test for any endpoint that returns data. A simple way to verify the structure might be to use json schema validation like so:

var expectedResponseSchema = {
    "type": "object",
    "properties": {
        "id": {
            "type": "string"
        },
        "blockId": {
            "type": "string"
        }
    },
    "required": ["id", "blockId"]
}
var result = client.get("/transactions/get?id=12345");
jsonSchemaValidator.validate(result, expectedResponseSchema);

Few months back I bought a small amount of coins (107) from bittrex now I find that my wallet has zero balance and the address is completely different.
Can anything be done?

Address: AN5JpGaKt1eiDFsNS6Tz9ctViymoc81XJ2

Not sure if this is possible, but is there some way to encrypt the secret in the config file? I just don't like the idea of seeing my secret in the plain text file, what if the server got hacked?

createGenesisBlock.js
createGenesisBlockFromAccounts.js

Change signing to use ark-js?

Having a success: false property in the response is not RESTful.

It would be faster (less bandwidth, response parsing) and more compliant (if we were aiming for a REST interface) to use standard HTTP status codes with an optional error in some cases, 400, 404, etc.

node tasks/createGenesisBlock.js

returns: Error: ENOENT: no such file or directory, open './demo/genesisBlock.vixcoin.json'

I propose add this parametrs for /api/transactions:

1. minAmount, maxAmount, to set transactions amount
2. type, to choose type of transaction
3. smart bridge, get transaction with specify smart bridge value

For reference so I can take a look. My node stopped completely, logging the below message. It was still trying to receive blocks. Restarting Ark (R in ARKcommander) fixed the issue and it started syncing again.

[ERR] 2017-11-15 11:53:51 | stack - Error: Unable to set non-blocking to true
    at Client.dispatchQuery (/home/alex/ark-node/node_modules/pg-native/index.js:196:26)
    at Client.query (/home/alex/ark-node/node_modules/pg-native/index.js:214:8)
    at NativeQuery.submit (/home/alex/ark-node/node_modules/pg/lib/native/query.js:128:17)
    at Client._pulseQueryQueue (/home/alex/ark-node/node_modules/pg/lib/native/index.js:171:9)
    at Client.query (/home/alex/ark-node/node_modules/pg/lib/native/index.js:130:8)
    at /home/alex/ark-node/node_modules/pg-promise/lib/query.js:135:27
    at promise (/home/alex/ark-node/node_modules/pg-promise/lib/promise.js:22:20)
    at Database.$query (/home/alex/ark-node/node_modules/pg-promise/lib/query.js:124:12)
    at Database.<anonymous> (/home/alex/ark-node/node_modules/pg-promise/lib/query.js:227:23)
    at /home/alex/ark-node/node_modules/pg-promise/lib/database.js:238:42

KiB Mem :  1999040 total,   139376 free,   433004 used,  1426660 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  1330008 avail Mem

Filesystem      Size  Used Avail Use% Mounted on
udev            968M     0  968M   0% /dev
tmpfs           196M   13M  183M   7% /run
/dev/sda1       9.7G  5.7G  4.0G  59% /
tmpfs           977M  4.0K  977M   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           977M     0  977M   0% /sys/fs/cgroup
tmpfs           196M     0  196M   0% /run/user/1001

root@vps456227:/home/alex# pg_isready
/var/run/postgresql:5432 - accepting connections
root@vps456227:/home/alex# /etc/init.d/postgresql status
● postgresql.service - PostgreSQL RDBMS
   Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
   Active: active (exited) since Wed 2017-11-15 07:00:15 CET; 5h 58min ago
  Process: 23772 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
 Main PID: 23772 (code=exited, status=0/SUCCESS)
    Tasks: 0
   Memory: 0B
      CPU: 0
   CGroup: /system.slice/postgresql.service

Nov 15 07:00:15 vps456227 systemd[1]: Starting PostgreSQL RDBMS...
Nov 15 07:00:15 vps456227 systemd[1]: Started PostgreSQL RDBMS.

For now this query for correct data return: success,address,unconfirmedBalance,balance,publicKey,unconfirmedSignature,secondSignature, secondPublicKey, multisignatures, u_multisignatures

My idea is to add for this result also delegate array for which this account vote. I think it isn't problem as long as one account is able to vote for one delegate.

error: pathspec 'newtestnet' did not match any file(s) known to git.

"dependencies": {
"arkjs": "https://github.com/arkecosystem/ark-js#newtestnet",

is this right link ?

missing the second s in ecosystem in the url

I noticed that the majority of the "Blockchain not ready to receive block" messages happens at, or very close to, block numbers that can be divided by 51. That message brings the weaker of my servers to it's knees several times a day, so I have to restart app.js. Just as heads up that there seems to be a point of heavy load on the servers at the round turnover.

It looks like we can already run two nodes which secrets plugged in. It is as simple as stopping forging node and restarting backup node.

It would be cool if we can add a failover functionality in order that backup relay nodes would automatically pickup forging if main delegate node fails.

It would be nice if in the results of the API call:
api/transactions/?type=3
the votes array returned not only the address, but the delegate name as there should always be a delegate name

Not sure if this is the right place to post but I think you should change how the voting system works. Able to add only 1 delegate is very frustrating. What if I have 100k ARK and want to split them between 10 delegates? I should create 10 accounts and remember 10 passphrases??? Don't think so.

I believe the right approach is to be able to add as many delegates as you want and just assign a % weight of your balance to every delegate. This way you could add 10 delegates, assign 10% weight to everyone and that's it, every delegates will receive 10k voting power. No need to create multiple accounts.

Disappearing Transactions bug

Hello Ark-ers, I've been digging through some bugs in some DPOs coins.

Lately I've discovered a bug in LISK and SHIFT were rounds were not properly rolled back possibly causing some network outages. (You can read more about it here & here)

Some guys asked me to assert that ARK was not affected so I did my debugging on ARK core too.

I can assert that the same race conditions that produced the bug in other coins does NOT affect ARK as well. BUT I've found another bug that is related (but more likely to happen) to the above mentioned one.

What is the severity of the bug?

Have you ever experiences disappearing txs from the network? I may know why!

When the bug occurs, one or more already confirmed transactions may disappear! This may cause irritation between users or problems when withdrawing/depositing from/to exchanges.

Does this mean that some accounts have more/less coins as a result?

No. Disappearing txs are properly rolled back so the sender/receiver's balances are restored as the tx never happened.

When does this happen? What are the causes?

To make it happen we just need the following elements that combined create the bug 'recipe':

• an active delegate
• with more than one node
• double forging
• one or more of its node with all ports inboud connections (optional, may happen - even if way more unlikely - even if all the nodes are reachable through the internet )
• double forged block has certain charateristics. (see below)

Ok so what may happen exactly?

Lets say Bob has 2 nodes (A & B (firewalled)) and the next forging slot is Bob's. If Bob double forges this is what may happen:

• A receives trough transports API several tx to include in its forged block
• B does not (node has only outbound internet connectivity and fails to pick txs up through transport API)
• A forges its block (with txs)
• B also forges its block (w/out any tx)

Now what my nodes see is the following:

• Some pending txs
• A blocks (with ^^ txs)
• B blocks (without txs)

When both blocks arrive, onBlockReceived event is casted through the event bus which will perform a series of checks in modules.nodeManager to assert the block is valid.

When A block arrives the txs previously received are correctly included within the block so they're removed from the pool as they are now confirmed; lately, when block B arrives it would have the same values of the A block except from the transactions field, and, of course, id and signature which will be different (since A's block getBytes will include txs as well).

If Bs block have a lower id than A's block, then swapLastBlockWith is called causing a rollback of all the things that were "confirmed" within A's block.

It's here where the code is buggy. The rollback functionality shouldn't just rollback but also re-insert txs in the queue/unconfirmed transactions pool allowing next (good citizen) forger to re-pick them up reconfirming them by including all the txs in the newly forged block.

PoC Code

Since the bug is strange and hard to replicate I took the liberty to write a piece of code that basically simulates the recipe showing that the txs are, indeed, deleted for good.

PoC code requires node 7 and it's basically app.js with some minor modifications at the top and the bottom of the file. Code is available here

I also used docker to test against a new pg db that would let me truncate and restore db easily. Here the command if you want to easily test. ->

docker run --rm -p 5432:5432 -e POSTGRES_USER=ark_test -e POSTGRES_DB=ark_test -e POSTGRES_PASSWORD=password --name pg postgres:9.6.3-alpine

Be aware that PoC code will use test/ folder config & genesisBlock file since the code needs passphrases to craft blocks and txs.

How to fix?

When doing a rollback we would just need to reinsert, the now, unconfirmed txs into the transactionsPool.

If I've time I'll try to do it later today. If you've any question I'll be in slack @vekexasia.

A different responses for block... i.e. calling /peer/blocks/?lastBlockHeight=1505795 I get
"totalAmount": "0", "totalFee": "0", "reward": "200000000",

when getting from other nodes using POST /peer/blocks the values are in int. And according to schema INTEGER is OK.
"totalAmount": 0, "totalFee": 0, "reward": 200000000,

Suggesting to adjust this in next release.

    # Looking for ark-node installations and performing actions
    arkdir=`locate -b ark-node`

This locate might return more than one folder and ARKcommander does not work anymore.

Example:
jarunik@vps416986:~$ locate -b ark-node
/home/jarunik/ark-node
/home/jarunik/work/src/github.com/jarunik/ark-go/cmd/arkgoserver/log/goark-node.log

You will get a ARKcommander.sh: line 1078: [: /home/jarunik/ark-node: binary operator expected error and you can't start the node.

Most likely the search for ark-node should be a bit more restrictive.

https://github.com/ArkEcosystem/ark-node/blob/447817b22f06c447bcc95358247e123da850c815/sql/blocks.js#L16

It would be better to wrap many of these queries into functions with SECURITY DEFINER so that direct table access can be locked down to only the super user. Those functions can move to migrations folder along with table and view creations. I would be happy to fork and pull-request that effort if it were welcome. I'd also semantically version the migrations folder to simplify updates. Please let me know :)

Example: http://178.32.203.37:4000/api/peers?limit=1&offset=10000

The ark-js submodule refers to a non-existent commit. Clicking through on Github results in a 404 error.

$ git submodule init
Submodule 'test/ark-js' (https://github.com/arkecosystem/ark-js.git) registered for path 'test/ark-js'
$ git submodule update
Cloning into '[HOME]/src/ark-node/test/ark-js'...
error: Server does not allow request for unadvertised object d5600d8d974fe296ef3323c01bb0720839a4621d
Fetched in submodule path 'test/ark-js', but it did not contain d5600d8d974fe296ef3323c01bb0720839a4621d. Direct fetching of that commit failed.

Git shows the commit doesn't exist.

$ git show d5600d8d974fe296ef3323c01bb0720839a4621d
fatal: bad object d5600d8d974fe296ef3323c01bb0720839a4621d

This is protocol level issue:

• fast fix: prevent from node to start if block height 1 in db is different from genesisBlock.json
• full fix: upgrade protocol, for instance signing all blocks including nethash. Other protocol upgrades possible. Good for AIP proposal

I get this when starting a private network on an empty database:

node app.js -c tasks/config.json -g tasks/genesisBlock.json

it happens after applying a number of transactions.

ARKcommander.sh fails to run node if there is any directory including ark-node other than the ark-node.

locate -b ark-node will find all directories.

It should be changed to:

locate -b "\ark-node"

This will prevent the search to add wildcards and look for ark-node

DB schema does not populate appropriately. When I check the DB itself the genesis block isnt being inserted.

ark_testnet=> select * from blocks;

 id | rowId | version | timestamp | height | previousBlock | numberOfTransactions | totalAmount | totalFee | reward | payloadLength | payloadHash | generatorPublicKey | blockSignature | rawtxs
----+-------+---------+-----------+--------+---------------+----------------------+-------------+----------+--------+---------------+-------------+--------------------+----------------+--------
(0 rows)

inf] 2016-11-25 16:21:32 | Ark started: 0.0.0.0:4000
NOTICE:  view "blocks_list" does not exist, skipping
NOTICE:  view "full_blocks_list" does not exist, skipping
NOTICE:  view "trs_list" does not exist, skipping
NOTICE:  trigger "protect_mem_accounts" for relation "mem_accounts" does not exist, skipping
NOTICE:  function revert_mem_account() does not exist, skipping

[dbg] 2018-03-05 09:27:58 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:27:59 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":221,"stale":true,"rebuild":false,"height":1}
[trc] 2018-03-05 09:28:02 | requestFromPeer - {"ip":"116.85.49.217","port":4100,"errors":31,"status":"ETIMEOUT","delay":0}
[trc] 2018-03-05 09:28:07 | requestFromPeer - {"ip":"47.91.218.198","port":4100,"errors":32,"status":"ETIMEOUT","delay":0}
[dbg] 2018-03-05 09:28:07 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":229,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:28:08 | Removing from memory blockchain blocks with height under - -199
[trc] 2018-03-05 09:28:12 | requestFromPeer - {"ip":"47.91.218.198","port":4100,"errors":33,"status":"ETIMEOUT","delay":0}
[dbg] 2018-03-05 09:28:15 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":237,"stale":true,"rebuild":false,"height":1}
[trc] 2018-03-05 09:28:17 | requestFromPeer - {"ip":"47.91.218.198","port":4100,"errors":34,"status":"ETIMEOUT","delay":0}
[dbg] 2018-03-05 09:28:18 | Removing from memory blockchain blocks with height under - -199
[trc] 2018-03-05 09:28:22 | requestFromPeer - {"ip":"47.91.218.198","port":4100,"errors":35,"status":"ETIMEOUT","delay":0}
[dbg] 2018-03-05 09:28:23 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":245,"stale":true,"rebuild":false,"height":1}
[trc] 2018-03-05 09:28:27 | requestFromPeer - {"ip":"116.85.49.217","port":4100,"errors":32,"status":"ETIMEOUT","delay":0}
[dbg] 2018-03-05 09:28:28 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:28:31 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":253,"stale":true,"rebuild":false,"height":1}
[trc] 2018-03-05 09:28:32 | requestFromPeer - {"ip":"47.91.218.198","port":4100,"errors":36,"status":"ETIMEOUT","delay":0}
[dbg] 2018-03-05 09:28:37 | peers validation error - ETIMEOUT Request failed GET http://47.91.218.198:4100/peer/list
[dbg] 2018-03-05 09:28:38 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:28:39 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":261,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:28:47 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":269,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:28:48 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:28:55 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":277,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:28:58 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:29:03 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":285,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:29:08 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:29:11 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":293,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:29:18 | Removing from memory blockchain blocks with height under - -199
[dbg] 2018-03-05 09:29:19 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":301,"stale":true,"rebuild":false,"height":1}
[dbg] 2018-03-05 09:29:27 | Blockchain state - {"date":"2018-03-05T09:24:18.921Z","secondsAgo":309,"stale":true,"rebuild":false,"height":1}

found a minor bug in the way transactionInPool is coded.

... edited ... irrelevant

Running npm install or yarn after freshly cloning ark-node results in:

> [email protected] install /Users/${USER}/src/ark-node/node_modules/libpq
> node-gyp rebuild

/bin/sh: pg_config: command not found
gyp: Call to 'pg_config --libdir' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:336:16)
gyp ERR! stack     at emitTwo (events.js:125:13)
gyp ERR! stack     at ChildProcess.emit (events.js:213:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Darwin 16.7.0
gyp ERR! command "/usr/local/Cellar/node/8.5.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/sumantmanne/src/ark-node/node_modules/libpq
gyp ERR! node -v v8.5.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

On macOS this is resolved with brew install postgresql.

Fix

An informative error should be displayed if no postgresql installation is found.