dtube / avalon

Blockchain for social distribution

AVALON

Get a node running

Dependencies

• MongoDB
• NodeJS v14/16 (LTS)
• ntpd or any NTP alternative for your system. ntpd comes pre-installed on most linux distributions

Install and run an Avalon node for Linux

• npm install to install nodejs dependencies
• chmod +x scripts/start.sh
• ./scripts/start.sh

Environment Variables

The start.sh shows the list of available environment variables you can set to make avalon behave slightly differently from the default install.

Install and run an Avalon node for Windows

• npm install to install nodejs dependencies
• Get your own keys with node src/cli.js keypair
• Save your keys
• Add your keys to scripts/start.bat
• Define the path to your directory in scripts/start.bat
• Run scripts/start.bat
• Note: to restore a genesis.zip file you may need to download the mongo databasetools and put the mongorestore.exe binary into your main directory.

Syncing your node

Become a leader and produce blocks

Debian 10 quick install procedure

Get helped

We have a discord channel dedicated to node owners (aka leaders), where you can get support to get set up. Join discorg.gg/dtube and go to DTube Chain -> #leader-candidates

Using Avalon

Once you have a node running, there are many ways to interact with Avalon:

With the CLI tool

You can use the CLI tool to transact with Avalon. Simply try node src/cli --help or node src/cli <command> --help for a full help.

Using Javalon

Javalon is the javascript wrapper for Avalon's API. Working on both browser and nodejs.

HTTP API

Avalon's API uses 100% JSON. The GET calls will allow you to fetch the public information which is already available through the d.tube UI.

Examples:

• Account data: /account/:username, i.e https://avalon.d.tube/account/rt-international
• Video data: /content/:username/:link i.e. https://avalon.d.tube/content/rongibsonchannel/QmdjVMdeTtTEy1CJTDbtjuaiRKMP6H364Dv4n7FsWGpnPH

Full list of API endpoints

https://docs.google.com/spreadsheets/d/1ORoHjrdq5V5OkTChijTUOEYRzTujVXTzCyNYt-ysVhw/edit?usp=drive_web&ouid=109732502499946497195

This lists all the available API endpoints for Avalon. We also have recommended security practises if you want to open your node's API to the world. You can do it easily with nginx and avalon-nginx-config.

Sending Transactions to the network (POST /transact)

Once you have an account and balance (if you don't, you can create one on https://signup.d.tube, your account will start generating bandwidth and voting power (respectively the bw and vt fields in your account data). You can consume those resources by transacting.

Every transaction will have a bandwidth cost, calculated based on the number of bytes required for the storage of your transaction inside a block. Certain transaction types will require you to spend voting power, such as publishing a content, voting or tagging a content.

To transact, you need to use the /transact POST call of the Avalon API. All the examples here are for the CLI tool, but the same can be achieved with Javalon in Javascript.

Necessary for all transactions:

• key: your private key
• • Use -K MyKeyHere to use a plain-text key
• • Or use -F file.json to use a key inside a file to sign (this will prevent your key from showing on the screen too much)
• user: your username

Vote for a leader

• target: the node owner to approve

node src/cli.js vote-leader -K <key> -M <user> <target>

// alice votes for bob as a leader
node src/cli.js vote-leader -K 5DPwDJqTvMuykHimmZxThfKttPSNLzJjpbNtkGNnjPAf -M alice bob

Unvote a leader

• target: the node owner to approve

node src/cli.js vote-leader -K <key> -M <user> <target>

// charlie does not want to vote for daniel as a leader anymore
node src/cli.js unvote-leader -F charlie_key.txt -M charlie daniel

Transfer tokens

• receiver: username of the receiver of the transfer
• amount: number of tokens to transfer to the receiver. Warning! 1 DTC in UI = 100 tokens
• memo: arbitrary short text content

node src/cli.js transfer -K <bob_key> -M <user> <receiver> <amount> <memo>
// bob sends 50 DTC to charles
node src/cli.js transfer -K HkUbQ5YpejWVSPt8Qgz8pkPGwkDrMn3XECd4Asn3ANB3 -M bob charles 50 'thank you'

Add a post / Commenting

• link: a short string to be used as the index of the content
• parent_author: the username of the author of the parent post
• parent_link: the link of the parent post
• json: arbitrary json input. example: {"string":"aye", array:[1,2,3]}
• tag: arbitrary short text content
• weight : the number of vote tokens to spend on this vote

node src/cli.js comment -K <key> -M <user> <link> <parent_author> <parent_link> <json>

Vote a post

• link: the link of the post to vote on
• author: the username of the author to vote on
• weight: the number of vote tokens to spend on this vote
• tag: arbitrary short text content

node src/cli.js vote -K <key> -M <user> <link> <author> <weight> <tag>

Edit your user json object

• json: arbitrary json input. example: {"string":"aye", array:[1,2,3]}

node src/cli.js profile -K <key> -M <user> <json>

Follow a user

• target: the user to follow

node src/cli.js follow -K <key> -M <user> <target>

Unfollow a user

• target: the user to unfollow

node src/cli.js unfollow -K <key> -M <user> <target>

Signing a raw transaction

To create a transaction and export it to a file, you can use the sign CLI tool

node src/cli.js sign <priv_key> <user> <tx> > tmptx.json

For example to approve a node owner and publishing it only 5 seconds later:

node src/cli.js sign -K 4L1C3553KRETK3Y -M alice '{"type":1,"data":{"target":"miner1"}}' > tmptx.json
sleep 5
curl -H "Content-type:application/json" --data @tmptx.json http://localhost:3001/transact

Other POST Calls

Mine Block

Will force the node to try to produce a block even if it's unscheduled. Useful for block #1 and working on development

curl  http://localhost:3001/mineBlock

Add peer

Manually force connection to a peer without having to restart the node

curl -H "Content-type:application/json" --data '{"peer" : "ws://localhost:6001"}' http://localhost:3001/addPeer

Data storage

MongoDB

Avalon saves the state of the chain into mongodb after each block. You can easily query mongodb directly to get any data you want, that wouldn't be provided by the API itself.

mongo <db_name>
db.accounts.findOne({name:'master'})
db.blocks.findOne({_id: 0})

However be sure not to write to any collection used by avalon in this database (namely the accounts, blocks and contents). If you do, your node will irremediably fork sooner or later.

Elastic Search

Avalon can also copy the accounts and contents into an elastic search database with monstache. A configuration file for monstache is provided in the root of this repository. Once running you can do text queries on accounts or contents like so:

# search contents
curl http://localhost:9200/avalon.contents/_search?q=football
# search accounts
curl http://localhost:9200/avalon.accounts/_search?q=satoshi

Please refer to Elastic Search documentation for more complex queries.

D.Tube runs a public Elastic Search mirror of the current testnet on https://search.d.tube

avalon's Issues

crash during sync using Memory-fix branch

everything worked well until it hit the HF. on block 1,200,081 see error below:

[2019-08-19T20:34:31.198] [INFO] default - block #1200000: 6 tx(s) mined by los-angeles dist: 18719 burn: 10753
[2019-08-19T20:34:31.484] [INFO] default - Hard Fork #1200010
[2019-08-19T20:34:33.669] [ERROR] default - Wrong dist amount 11445 18352
[2019-08-19T20:34:33.669] [ERROR] default - Error Replay { _id: 1200081,
  phash:
   '202677eed0d82010ea13b58efd32db3206e49df48434427864f4a3e8bacc092c',
  timestamp: 1565406055129,
  txs:
   [ { type: 5,
       data: [Object],
       sender: 'sanashaikh',
       ts: 1565406052829,
       hash:
        'ff7c87776ea891358f649158e5cca5b5420aca4248f775d6ff1b16710d8f91cd',
       signature:
        '22ADk4BVKNeMjq6ug5xcDv8qMiHY2CGA5x5rgHKQiqJvJjZEqtbkr261jA6pXuXvW2KK8TBV8oEp1u9zsshB9tp1' } ],
  miner: 'ivansnz',
  dist: 11445,
  hash:
   '39054908b739cd2f8e83e0882617ec642b1e2db0075d5cb058ff554ee17d86f4',
  signature:
   '4wtwHEbygR5E9JM7F7MPYan4cBbuKLKZsS1YVZQBYLLuCUwoZT5ukcRGXkueEcEi6D6viVRcF55x4sVzRT9SWWWL' }
[2019-08-19T20:35:16.066] [WARN] default - peer connection failed ws://185.178.193.56

Upon restarting the start.sh script, the error cleared itself.

Mongodb V4.2.0
node V10.16.3

Trying to get in touch regarding a security issue

Hello 👋

I run a security community that finds and fixes vulnerabilities in OSS. A researcher (@exmachinagpt) has found a potential issue, which I would be eager to share with you.

Could you add a SECURITY.md file with an e-mail address for me to send further details to? GitHub recommends a security policy to ensure issues are responsibly disclosed, and it would help direct researchers in the future.

Looking forward to hearing from you 👍

(cc @huntr-helper)

[Question] Can a username reset its password, without previous password being used?

For example, I create a account for a Twitch streammer. He joins DTube but shoudn't trust me having its privatekey. Can he change privatekey without the initial privatekey I have working?

Invalid timestamp when transacting with a bad system clock

Avalon will refuse transactions with bad timestamps for the transaction. Some users have bad clocks but shouldn't be blocked from using the UI. Instead, we could look at the timestamp of the previous block and add ~3 secs, and use that for transacting on PCs who have unsyncd clocks.

Monstache middleware in golang

The search engine is struggling since because of the 'tags' object in the contents collection. Elastic search creates an index for each new tag used on DTube, and it's not a good practise for elastic search data (increased ram usage for no reason).

So we need a monstache middleware to convert the tags object into a normal array, before it gets added to elastic search. Then some changes in UI will be required too to make the tags navigation work again.

Create a new call for transacting that returns only when the transaction is included in a block

Presently, the /transact call of the avalon api, instantly returns after adding the transaction to the pool (if it seems valid on the current timestamp). Then, javalon does some monkey business (checking /block/X+1 repetedly on the API) to see if a transaction went through or not and being able to confirm that the transaction was a success to the UI.

In practise this works, but we could have another call /transactAndWait that waits for the transaction to been included in a block before returning. This would reduce the perceived time it takes to do any type of transaction through a UI by some good amount.

New accounts starting with ressources

Currently, the chain account creation is pretty simple. The user burns some coins, the account gets created with 0 ressources. The user needs to transfer some DTC and wait for ressources to grow.

I think we should somehow take a % of the burn, convert it to VP (maybe a % of the vtPerBurn config) and Bandwidth (not sure how to calculate it), and put it inside the new account created.

Just a random idea and requires some discussions

DTC enters Polygon (alongside BSC)

Sorting options for /blog endpoint

Is there a way to change the order of the videos on the channel page when viewing it publicly? I notice that the videos are posted in the order of the posted date. I have videos that are from YouTube that are older than my newest videos, but are in front.

Multi-signature accounts

Currently, all Avalon transactions are signed with only one private key, which makes it impossible to split account authorities across multiple keys potentially owned by different people.

We can add the ability to turn an account into a multisig account where a threshold is assigned to each transaction type, plus a default threshold if not defined. Each public key can have a weight assigned, which can be custom or master keys. Transactions signed by multisig accounts will contain more than one signatures in an array (containing signature data and secp256k1 signature recovery id), instead of a plain string.

Avalon doesn't detect it's on old nodejs (v4) and starts

I've tried to start and restart this several times, but it's not able to get past this block.

01:54:19.806 [ERROR] Wrong dist amount 12 11
01:54:19.807 [ERROR] Error Replay { _id: 29774,
  phash:
   '6a69d2d518becedbe448596bdbe7e3a084a0c025ce113d01f507b370b0014518',
  timestamp: 1561493332897,
  txs:
   [ { type: 5,
       data: [Object],
       sender: 'ignacioarreses',
       ts: 1561493330334,
       hash:
        '390d1a85f30e63a56d2464d8356d91efd7cee3c749603e0e23f0cf9484e9da9d',
       signature:
        '2886AuCRihgDT1k1ThfbLUEoMpGCjLkC2EQErzYoTPfF8CmwZRXKgn4WJS7UqyBNWz6Dm2Hr7h2ZN9RJRMshTQJM' } ],
  miner: 'wehmoen',
  dist: 12,
  hash:
   '05c9b84cf4d523a4e761d9851f0d1444e5f9f2aabf8496ef738327fbc834f130',
  signature:
   '5cQFFWqaLu5JmxuNkMUBmMBNxZh8HuMXXgHKcvX4THQJXHaNk7kasHqt7ufuYBQknfnA1Ljh4gAiRu95bRzCoctm' }```

I put a message on a post too but this is probably a better spot for it.

Add a tipping option to the vote transaction

Since we launched the chain for real, and coins have (seemingly increasing) value, it became hard for minnows to earn their few first DTCs with curation rewards.

We already have a tipping button that does the same as a normal transfer, but we all know that giving away money for free is psychologically difficult in the world we live in.

The way I see it is, I wouldn't mind donating my curation rewards to the author of the video sometimes. I see a lot of legit minnows posting contents with <100 VP, and when I vote it with my @heimindanger account, I give them 0.01 at best, while printing a few whole coins to myself... Somehow I wouldn't mind my curation rewards going to the author when I do that.

I believe we could add a isTip boolean option to the VOTE transaction. The only difference for those votes, is that instead of being claimable by the voter, it would be claimable by the author of the video. Implementation in the chain should be pretty straightforward.

Please let me know if you see any fallacy with such an option, otherwise I think it would be a good improvement to fix the issue of the legit minnow accounts struggling to earn coins.

Manipulable inflation

Avalon calculates inflation based off the number of total accounts in existence. A potential attacker could burn/create a number of accounts inorder to artificially increase inflation. That then said attacker could use each of those accounts to vote specific content that has a high likely hood of getting upvotes. (For example a popular, well know content creator). That then creates an artifical popularity of content that is being boted (an entire separate issue). Bots have the ability to vote on content way before a lot of people have seen it. (Thus making it potentially profitable)

Overall if this issue becomes wide spread it will make the chain worthless.

/feed and /new with an additional filter on tag

There should be additional endpoints for /feed and /new to filter on a tag.

There is the plan to implement a story/shorts feature inside of the app. For that I will need a new endpoint on the api level to load posts from followings with a specific tag included. ( I would suggest "moments" but making it dynamic is probably the best choice)

The same would be good to have for the /new endpoint. That way we could implement tag lists.

@skzap already had this idea:
3. is pretty easy to do honestly, just copy paste the /feed endpoint and add the filter by tag in the mongo query

Detailed error message about VP

Have been seeing users getting invalid tx.data.vt errors, it would be useful to indicate the details of the error. This can indicate the number of VP someone is trying to spend but only have a certain amount available like this:

Buyback policy for DTC

"Cannot read property 'suffle'" after restarting node

From tiem to time I stop my leader node and the Mongo server. After a few hours I restart them and I am consistently getting this TypeError which causes the start.sh to stop. When I try again, it just works.

Here I paste the log:

mac-mini:~ javi$ mongod --config /usr/local/etc/mongod.conf &
[1] 828
mac-mini:~ javi$ cd avalon
mac-mini:avalon javi$ ./scripts/start.sh
20:58:11.157 [INFO] Logger initialized
20:58:24.739 [INFO] Correctly using NodeJS v10.15.0
20:58:24.814 [INFO] Connected to mongodb://localhost:27017/avalon
20:58:25.003 [INFO] #1977747 is the latest block in our db
20:58:25.150 [INFO] Listening websocket p2p port on: 6001
20:58:25.157 [INFO] Listening http on port: 3001
20:58:25.159 [INFO] Generating schedule... NRNG: 11896451
20:58:25.383 [DEBUG] Handshaking new peer ws://35.203.37.221:6001
/Users/javi/avalon/src/chain.js:383
if (chain.schedule.shuffle[(newBlock._id-1)%config.leaders].name === newBlock.miner)
^

TypeError: Cannot read property 'shuffle' of null
at Object.isValidNewBlock (/Users/javi/avalon/src/chain.js:383:28)
at Object.validateAndAddBlock (/Users/javi/avalon/src/chain.js:131:15)
at Object.addRecursive (/Users/javi/avalon/src/p2p.js:266:15)
at WebSocket.ws.on (/Users/javi/avalon/src/p2p.js:143:25)
at WebSocket.emit (events.js:182:13)
at Receiver.receiverOnMessage (/Users/javi/avalon/node_modules/ws/lib/websocket.js:800:20)
at Receiver.emit (events.js:182:13)
at Receiver.dataMessage (/Users/javi/avalon/node_modules/ws/lib/receiver.js:422:14)
at perMessageDeflate.decompress (/Users/javi/avalon/node_modules/ws/lib/receiver.js:379:23)
at _decompress (/Users/javi/avalon/node_modules/ws/lib/permessage-deflate.js:298:9)
at _inflate.flush (/Users/javi/avalon/node_modules/ws/lib/permessage-deflate.js:377:7)
at afterWrite (_stream_writable.js:480:3)
at onwrite (_stream_writable.js:471:7)
at InflateRaw.afterTransform (_stream_transform.js:94:3)
at Zlib.processCallback (zlib.js:619:8)
mac-mini:avalon javi$ ./scripts/start.sh
20:58:32.038 [INFO] Logger initialized
20:58:32.824 [INFO] Correctly using NodeJS v10.15.0
20:58:32.869 [INFO] Connected to mongodb://localhost:27017/avalon
20:58:32.890 [INFO] #1977747 is the latest block in our db
20:58:32.952 [INFO] Listening websocket p2p port on: 6001
20:58:32.959 [INFO] Listening http on port: 3001
20:58:32.961 [INFO] Generating schedule... NRNG: 11896451
20:58:33.089 [WARN] peer connection failed ws://185.178.193.56
20:58:33.122 [DEBUG] Handshaking new peer ws://139.59.209.189:6001
20:58:33.135 [DEBUG] Handshaking new peer ws://34.65.181.103:6001

MongoError: E11000 duplicate key error collection: avalon.blocks

There is a bug for active leaders where sometimes it tries to mine at the same time a block is validated through consensus. Example: http://dpaste.com/1WKRPMN

We can see in the example the node was extremely slow to validate the block (0.8 secs) and probably tried to self mine while the approved block was already adding, creating this error and destroying the state in the db, forcing a replay...

Image Proxy in the Avalon API

I currently run image.d.tube (https://github.com/dtube/imagehoster) which is a fork of the steemit image proxy converted to use an avalon API instead of steem.

It's unstable, and frequently crashes the VM it's running on. It has a bunch of unused code (such as upload feature) that we do not use. We only use the /avatar and /cover endpoints for avatar and channel cover images.

I want to move this into avalon directly, probably will be off by default and be turnable on with API_IMAGE_PROXY=1, then nodes with this turned on will have the /avatar and /cover endpoints working the same way as they currently do.

If someone want to do it, feel free to drop a comment, otherwise I will do it probably in the coming weeks.

Avalon IPFS Pinning Script

https://ipfs.io/ipfs/QmfSgV1dVQ7PoQDyw1vePNCeUbBsKHYxoJtc636s1Y1fWM?filename=pinFollowing.sh

This script pins all videos from either: a list of people an account follows, or videos from a specific user

Example:
./pinFollowing.sh 240 true false devdeckardcain94

Where 240 is the quality I desire to pin (240, 480, 720, 1080), the second parameter determines whether I wish to pin (true) or just output the CID's (false), the third parameter is to determine what we should do with the fourth (true == load the following list from the user, false == just load the list of videos from this single user) and finally parameter four which is the account you're targeting.

Video tut: https://d.tube/v/devdeckardcain94/QmWcX9utpvJmp3Qa7fa1dFoxuk9k5c8jbjnjwM26zKstKh

Disable DTC generation on a video

I run an archive and post content that I don't want to monetized. Even though I have unchecked the "Original content" button, I still earn DTC.

Suggested fixes include:

1. Disable DTC generation for that video (hard)
2. Opt-in disable payout for this account option

Crashed - Using REST Endpoints

/usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/utils.js:113
throw err;
^

TypeError: Cannot read property 'follows' of null
at /usr/home/deckardcain/Documents/avalon/src/http/feed/index.js:28:34
at /usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/utils.js:704:5
at executeCallback (/usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/operations/execute_operation.js:65:7)
at handleCallback (/usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/utils.js:109:55)
at /usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/operations/find_one.js:31:9
at /usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/utils.js:704:5
at /usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/cursor.js:252:9
at /usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/core/cursor.js:746:9
at handleCallback (/usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/core/cursor.js:32:5)
at /usr/home/deckardcain/Documents/avalon/node_modules/mongodb/lib/core/cursor.js:690:38

Avalon Chain Halt Reason/Solution documentation/discussion

These are the known reasons for avalon chain halt

1. Offline: Less than 2/3 online (2/3 offline) with good chain state
   Is it possible for a node to know.. it's in "good chain state" or a bad chain state. If you are not producing for 3 * 15 = 45 sec or last 120 sec, then possibly you are in "bad chain state"

2. Double production: When a single leader with same leader key tries to mine from multiple device, it causes two different block hash generated in different device and broadcast.. other leader gets confused and some receive from one source.. other receives from other source for the same leader.. thus chain split.

3. Time limit exceeded: Very slowed down system (for disk usage for example) that slows down a leader's mining action, then the node tries to mine very late at the same time as the backup leader (the previous one). Forking the chain. It happened recently.

Add more reasons.. discuss possible solutions.

Possible solution:
Rollback if hasn't mined for more than 2 minute. Check whether mining becomes normal. Rollback at least 5 times. If it is still stuck, then notify.

Program halt

Had an issue with my internet dropping out (regular occurrence for the time being. ISP infrastructure is to blame), which caused this problem:

15:23:49.285 [ERROR] unauthorized miner
15:23:49.285 [WARN] miner worker trying to mine but couldnt Block {
_id: 9000757,
phash: '227c4ca3f8ceb13be6f39efd29b4aca86dd72280e625d6721d03586707236ff0',
timestamp: 1629919429284,
txs: [],
miner: 'devdeckardcain94',
hash: undefined,
signature: undefined
}

Once that error comes up, the program halts. It's still running, just not doing anything.

Error: invalid signature

Hello guys. I am getting an error declaring myself as a node leader using the below line:

node src/cli.js profile -K -M '{"node":{"ws":"ws://yourip:yourport"}}'

This is the line that I am executing along with the error that I am getting:

I just did this today, 12/25/2019!

I am using the same pub/private keys that I used to log into d.tube in the start.sh file.

Genesis.zip requires at least 4Gig

This happens on a 2 gig node. Works fine on a 4gig node.

Its still weird because the genesis.zip file is only 65MB.

Self commenting produces notification

If you comment on your own comment, that produces a notification which probably is not necessary.

Replay slows down

Doing a full replay from scratch starts off pretty fast but slows down over time because of poor memory management. Restarting the node periodically fixes the issue but it is dirty. Avalon needs to more properly manage memory during a replay.

Burning DTC for promo

I burnt 5 DTC, though it doesn't seem to have much effect. I'm not sure what the formula is for determining how big a boost you're going to get for burning DTC, but I feel as though it's a big ask.

The reason I say this is, with burning 5, I can't tell that any significant change has happened because of it. For me, idc, but for the end users I think they would.

Suggesting:

• Rework the burn DTC option to be a little more powerful (and hopefully scale with inflation so we don't have to fix this again)

Avalon contents NFT

It would be a great way to burn some tokens if we would sell custom video URLs within the uploader instead of the hash based URL. Especially for long-term content that would be nice to have

Sharding

As Avalon will have other social networks alongside DTube, what about one sidechain for each, while Avalon is the core?

Display leader age/lifespan on leaders page

Show leader lifespan/age on leaders page like the witness age on hive. May be also on the block explorer.

. I believe, it helps regular new dtubers, to decide on whom to trust and vote. May be in future, it can be an interesting variable in governance.

Clustering for secp256k1 signatures verification

Following #20 where I've been able to remove a bottle neck and increase the replay speed, I believe the next big step will be to enable multiple CPU cores usage for the heavy-weight operations that avalon does. So basically I want to do it for the signatures verification, which represent the ultra-majority of the CPU cycles used, and should improve the replay speed a lot for people with 3 cores or more. 2 cores won't change much because mongodb's process is usually already clusterized and will make use of the 2nd core already.

This would greatly improve the replay speed, especially for people using scalable cloud solutions, where it would then be possible to increase your CPU to replay faster, and turning it down once synced.

Invalid block

Hi, I got the following error,

14:52:13.171 [ERROR] timestamp from the future 1621848133527 1621848133171
14:52:13.173 [ERROR] Invalid block
14:52:13.175 [ERROR] Error Replay 6423899 

REST API Version & Avalon version seperation

The way that I'm going about being compatible allows for backwards compatibility. Instead of having the REST API version tied to Avalons version, can we have one for each?

Avalon 1.5 running REST API 1.0, or something like that. This way I can create separate implementations for each version.

Accounts don't have a creation date

Check for validity of announced IP of Avalon node leader

DTube Avalon uses leaders which are the equivalent to Steem witnesses. They are listed on their own section. Those leaders should add their own IP -or hostname- so other nodes are able to connect to them. When hover the mouse over a leader which has provided a URL, the UI shows it.

It is currently not checking for the IP address to be a valid one. Web page will show that leader as enabled -not grayed- but nodes will be unable to connect to it.

Leader Extractable Value (LEV)

A variant of Maximal Extractable Value (MEV) that could be extracted from curators voting on potentially underrated contents. Leaders involved in LEV activities would insert their own votes of the same direction into their own blocks (bypassing the mempool) when an incoming vote operation is observed such that the LEV votes will execute before the curator's vote does.

This takes advantage of the vote ordering that is being used to determine curation rewards as votes that it favours votes executed earlier. In this instance the curator is still under an impression that the content being voted on is underrated as the curator does not see the LEV votes that are being executed before the curator's vote resulting in lower rewards for the curator and higher rewards for the LEV votes.

There is a risk in LEV such that the content could be voted in the opposite direction afterwards, reducing rewards for LEV votes. In the case of the LEV vote being an upvote however, this is not a good mitigation as the later downvotes would directly affect rewards of author vote and any author tips unnecessarily. There is no guarantee that the downvoters would be nice to downvote with author tip.

Mitigation: Instead of benefiting all previous active votes for every new incoming vote, group the votes by the age of the content when the vote was made (such as 0-24 hours, 25-72 hours, 72-168 hours and 169 hours onwards). All votes in a group benefit from votes in the later groups equally. This retains the advantage of voting before others do while making LEV more difficult. It may also come with an added benefit of keeping the curation reward distribution easier to understand for the average user.

Unclaimed airdrops burn percentage

It's only been 2 months out of 6, but so far, only about 30% of the airdrop amounts have been claimed in these two airdrop periods.

I keep an history of what has been distributed and no distributed every month in the dtube/signup repo:

https://github.com/dtube/signup/tree/master/airdrop_history

The last distribution month will be March 2021, so we need to decide what to do with the final unclaimed airdrops which will coincidentally happen on April 1st 2021. At the current rate, it should represent about 1.4M DTC, a hefty sum.

I propose that we first settle on the % amount that we will burn. If you have ideas about what to do with the coins we will not burn, please keep it on the discord channel, or create a new issue in this repo. First we should settle on the burn amount, then we will discuss about what we do with the remainings tokens.

Integrate with Polkadot by implementing Substrate

https://polkadot.network/

Organize the HTTP API

The src/http.js (HTTP API) is starting to be huge. Also everyone more or less added some calls for his own purposes (see techcoder extended API).

I think it would be nice to split the API into multiple files, and why not modularize it so that anyone can add/remove each endpoint easily.

examples:

• the /block endpoint code would be in src/http/block/index.js
• the /rewards/* endpoints codes would be in src/http/reward/index.js, each endpoint could have it's own file in src/http/reward/

Avalon would by default use all the files in the src/http/ folder. Removing an endpoint would be as easy as deleting a folder. Adding an endpoint could be done via git submodule.

Idea

I have an idea which I posted on the discord and a user specifically suggested that I post it here.

quoting myself:

have you guys heard of bitclout?
what if channels had an associated coin with every video
they bet on the popularity of the video
and trade it like a coin
that can be traded with the channel's coin
which then can be traded radially
so viewers would be able to effect a sorting mechanism while giving creators an incentive to create content
bet big on that cat video?
second phase is to have metadata feedback for individuals who have a representation of people following them
third phase is a tor shell navigator

unquote

The market is hot for this sort of thing, content creators are under pressure by large corporations and often censor their work significantly. Since Robinhood, Dogecoin, etc. social engagement is like a hot pan.

To be paid you could blockify code functions for site coin.

There are many different monetization strategies that could make this successful

cosmicshockzen

Websockets hanging

It seems some people have an error around the node's consensus hanging rarely and randomly. According to logs it's because the node never reaches the 2/3+ threshold to pass consensus. I suppose that some websockets are left hanging (as if the internet cord was broke) and not properly terminated and re-opened, which is kind of crucial in terms of consensus. If a node thinks an active leader is online, but isn't validating blocks, then we end up in this situation where an observer node just gets stuck, and an active leader node forks on it's own chain.

I'm creating a websocket-terminate branch where it's already tracking the websockets health. Can see data in /peers from API e.g. curl http://localhost:3001/peers | jq ".[].lastMessageTime"

I will try to verify the theory is correct (some hanged websockets should show up in /peers with very long lastMessageTime.

No protection against leaders missing blocks

When leaders start missing blocks (for various reasons), it creates a 3 sec delay whenever this leader is supposed to be next, and degrades the user experience.

This also makes estimating the hardfork time in advance almost impossible, which sucks for any type of announcement done to the community (no fixed date can be given), and sucks especially more for leaders who need to monitor nodes at the moment of the hard fork...

Tags should allow for spaces

In most every other video service site, the tags form into bubbles when there is a space, thus combining the multiworded string into one ("). We need to do the same, because right now DTube is only good at searching for one term, not two or even a mixture of the two.

GPLV3 Pl0x - License change

### TLDR (too long, didn't read);

MIT is fine is you're interested in commercialization and would want to, at some point in the future, branch off and take the code with you. My personal opinion is, I'm not cool with this. I don't want to spend hundreds of hours getting something somewhere so that other people can more easily start companies, but I'm also not opposed to other people making money, so long as they pull their weight.

### For those that prefer video format:

Comparing licenses:
https://d.tube/#!/v/devdeckardcain94/zgn7xdo88dh

Linus Torvalds' opinion:
https://d.tube/#!/v/itsdevdc94/33gqoaw1pir

Comparison between multiple licenses:
https://d.tube/#!/v/itsdevdc94/zp2psx3zidv

Others:
https://d.tube/#!/v/itsdevdc94/9v9p83kmew5
https://d.tube/#!/v/itsdevdc94/9v74ls0hu3e

### Overview:

This will be a long, and ultimately very important conversation. What I want to begin discussing is a change from the MIT license to either GPLV3 (preferred) or GPLV2. The reasons for this is, I personally don't want to write open source code so that someone else could take it and start a company.

It is in my belief that DTube should be communal, as in no one company should be able to just take our code, make it better and release their own DTube completely separate from us. I'm not opposed to making money, but I do think that if someone wants to make money off of something, it shouldn't be off the backs of the community.

The proposed change in license wouldn't restrict business form happening, but would instead protect the core of what we all value (the DTC and the webui). Should these two things be protect with a license which would ensure that anyone who makes changes must publish those changes. This would ensure that if some company came around and wanted to take what we have and expand upon it, they would be more than welcome to do so through the API using whatever licenses they want, but if they want to touch the core in any way (not using the REST API), then they must pull their weight and send those changes to us for approval / disapproval (we don't have to accept the changes they make, we just have to have the option of getting them).

An additional thing to consider when considering GPL, is the FSF (fsf.org), who will hire lawyers and fight on your behalf should someone violate the license.

By no means am I saying my way or the highway, perhaps through more dialectic we can decide on something altogether different, but I do feel this discussion is incredibly important as some developers don't like working on projects with certain licenses, so we need to be sure that the license we have is there for good reason.

Smart contract on Adax (Cardano chain) through ERC20 Conversion

I would like to convert dtube's ERC20 for Cardano and make it compatible with Adax (Uniswap of Cardano).

Reason1: Cardanos AM (automated maker) promises to reduce fees for trading pairs
Reason2: They will be launching in 2022 and we can be one of the few first coins to join and get marketing exposure

Adax : "ADAX is an automated liquidity protocol that facilitates trades within the Cardano ecosystem in a completely decentralized and non-custodial way. "
See more in details: https://adax.pro/

twitter: https://twitter.com/adax_pro

news: https://cointelegraph.com/news/one-decentralized-exchange-is-positioning-itself-as-the-uniswap-of-cardano-with-a-suite-of-cardano-tailored-defi-solutions

Account following/followers scalability issue

Currently, the full list of accounts followed and followers are stored in a single document in accounts collection. As MongoDB limits document sizes to 16MB, this could become an issue when a channel has millions of subscribers, where storing the entire list of account names will exceed this limit.

Also, the /account/:name API returns the full account details, which is all the data stored in the document. If the full list of accounts following/followers can be returned, the response could be several gigabytes in size for a large channel.

"request" library not installed but needed

When executing start.sh I am now getting this error message:

mac-mini:avalon javi$ ./scripts/start.sh
14:15:13.804 [INFO] Logger initialized

The "request" library is not installed automatically anymore.

But is a dependency of "request-promise".

Please install it with:

npm install request --save

My local copy of the repository is up to date. I also ran "npm install" after pulling the source code.

sync Avalon locally on my system

hii everyone I am cloning DTube and want to set up Avalon on my system for publishing videos. I want to sync Avalon locally on my system. How can I do it?