tendermint / coding

Tendermint

UPDATE: TendermintCore featureset is frozen for LTS, see issue #9972
This is the latest stable release used by cosmoshub-4, version 0.34.24
The previous main branch (v0.38.xx) can now be found under "main_backup"

Byzantine-Fault Tolerant State Machine Replication. Or Blockchain, for short.

Branch	Tests	Linting
main

Tendermint Core is a Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines.

For protocol details, refer to the Tendermint Specification.

For detailed analysis of the consensus protocol, including safety and liveness proofs, read our paper, "The latest gossip on BFT consensus".

Documentation

Complete documentation can be found on the website.

Releases

Please do not depend on main as your production branch. Use releases instead.

Tendermint has been in the production of private and public environments, most notably the blockchains of the Cosmos Network. we haven't released v1.0 yet since we are making breaking changes to the protocol and the APIs. See below for more details about versioning.

In any case, if you intend to run Tendermint in production, we're happy to help. You can contact us over email or join the chat.

More on how releases are conducted can be found here.

Security

To report a security vulnerability, see our bug bounty program. For examples of the kinds of bugs we're looking for, see our security policy.

We also maintain a dedicated mailing list for security updates. We will only ever use this mailing list to notify you of vulnerabilities and fixes in Tendermint Core. You can subscribe here.

Minimum requirements

Requirement	Notes
Go version	Go 1.18 or higher

Install

See the install instructions.

Quick Start

• Single node
• Local cluster using docker-compose
• Remote cluster using Terraform and Ansible

Contributing

Please abide by the Code of Conduct in all interactions.

Before contributing to the project, please take a look at the contributing guidelines and the style guide. You may also find it helpful to read the specifications, and familiarize yourself with our Architectural Decision Records (ADRs) and Request For Comments (RFCs).

Versioning

Semantic Versioning

Tendermint uses Semantic Versioning to determine when and how the version changes. According to SemVer, anything in the public API can change at any time before version 1.0.0

To provide some stability to users of 0.X.X versions of Tendermint, the MINOR version is used to signal breaking changes across Tendermint's API. This API includes all publicly exposed types, functions, and methods in non-internal Go packages as well as the types and methods accessible via the Tendermint RPC interface.

Breaking changes to these public APIs will be documented in the CHANGELOG.

Upgrades

In an effort to avoid accumulating technical debt prior to 1.0.0, we do not guarantee that breaking changes (ie. bumps in the MINOR version) will work with existing Tendermint blockchains. In these cases you will have to start a new blockchain, or write something custom to get the old data into the new chain. However, any bump in the PATCH version should be compatible with existing blockchain histories.

For more information on upgrading, see UPGRADING.md.

Supported Versions

Because we are a small core team, we only ship patch updates, including security updates, to the most recent minor release and the second-most recent minor release. Consequently, we strongly recommend keeping Tendermint up-to-date. Upgrading instructions can be found in UPGRADING.md.

Resources

Libraries

• Cosmos SDK; A framework for building applications in Golang
• Tendermint in Rust
• ABCI Tower

Applications

• Cosmos Hub
• Terra
• Celestia
• Anoma
• Vocdoni

Research

• The latest gossip on BFT consensus
• Master's Thesis on Tendermint
• Original Whitepaper: "Tendermint: Consensus Without Mining"
• Tendermint Core Blog
• Cosmos Blog

Join us!

Tendermint Core is maintained by Interchain GmbH. If you'd like to work full-time on Tendermint Core, we're hiring!

Funding for Tendermint Core development comes primarily from the Interchain Foundation, a Swiss non-profit. The Tendermint trademark is owned by Tendermint Inc., the for-profit entity that also maintains tendermint.com.

coding's Issues

Dist fuzzing - dist process should not be so deterministic.

A "distribution" event is a singular event with high ramifications (e.g. distributing a hacked binary will affect many people), we might as well take extra precautions while releasing a dist.

We could make the dist, signing, and releasing process automated, and the obvious attack is to hack the signing server or the code that handles the signing.

It's more secure to have many signers, for that mitigates the risk of getting the server hacked. But it doesn't mitigate the risk of getting the dist-building code itself hacked.

A partial solution is to not specify/automate certain things which could be fixed manually. For example, the dist script could choose not to specify a docker image to build under, but rather inherit the environment from the caller.

So the dist-builders are required to find an environment to build in, whether that be a Docker container or something else. This manual setup acts as a kind of fuzzer. There are many configurations that should lead to the same build, such as using Golang version 1.9.x

If we find a discrepancy, it might be because we found a vulnerability, so instead of getting annoyed, we should be thankful and learn what caused the unintended non-determinism, and isolate it, but only it.

I should figure out a way to propose things without using the word "should". I could say "could"...

document all repos and all binaries

• of the open-source stack - to orient newsers

When to use `package X_test`

package X_test forces you to do testing through the public interface. Most of our packages dont use it. When should they ?

consistency in cobra helper messages

e.g.

--option strings    Genesis option in the format <app>/<option>/<value> (default [stake/allowed_bond_denom/fermion])
--static            use a static private validator

lower case or Upper case?

Github label system

Priority (color #fad2fa)

• bug
• priority? - use to tag what might be priority

Type (color #d2d2fa)

• proposal - feature requests, enhancement, etc
• question
• task
• policy - process or policy issues
• milestone - an issue describing the milestone

Highlight (color #e6fad2)

• feature - something worth featuring in publications
• bounty - person doing the tagging, or representing some bounty program, will pay a bounty for it.
• help wanted - help wanted from anyone
• good first issue - good entry-level bug, to get familiar w/ the codebase

Concerns (color #fafad2)

• p2p
• rpc
• consensus
• ... project specific, go nuts

Meta (color #e6e6e6)

• duplicate
• wontfix
• fixed

Documentation (color #d2fafa)

• docs - need (updates to) documentation
• spec - need (updates to) specification

Dont use

• invalid -> use wontfix
• enhancement -> use proposal
• priority - high -> use priority? or bug or milestones (codeowner)

Only put a tag within the glide.yaml file

I recommend to only include specific tags/releases in your glide.yaml file. That is the only way that you can avoid having the old builds break when someone force pushes to your dependency.

If you are interested in a real world example, please call me :-)

deprecate `--root` from the stack

all: discourage panics in libraries/packages where possible

I would like to propose that we discourage panics within library/packages unless something is totally unrecoverable or if it is a sanity check. I've seen a bunch of panics in our libraries for functions with signatures that could be modified to return an error. IMHO packages should be the glue code that userland code invokes and shouldn't get surprised by; userland code includes CLIs, servers, handlers etc.
What do y'all think?

makefile standard

started here tendermint/abci#168

Catalog seldom used or unused functions

eg. IsDirEmpty (tendermint/tmlibs#34 (comment))

automate pushing docker images on release

• see this comment tendermint/tendermint#678 (comment)

• applied also to ethermint, and ... ?

linting: gometalinter

should be applied to all go code

use PR refs for easy code review

via @adrianbrink:

Locate the section for your github remote in the .git/config file. It looks like this:

[remote "origin"]

	fetch = +refs/heads/*:refs/remotes/origin/*

	url = [email protected]:joyent/node.git

Now add the line fetch = +refs/pull/*/head:refs/remotes/origin/pr/* to this section. Obviously, change the github url to match your project's URL. It ends up looking like this:

[remote "origin"]

	fetch = +refs/heads/*:refs/remotes/origin/*

	url = [email protected]:joyent/node.git

	fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

Now fetch all the pull requests:

$ git fetch origin

From github.com:joyent/node

 * [new ref]         refs/pull/1000/head -> origin/pr/1000

 * [new ref]         refs/pull/1002/head -> origin/pr/1002

 * [new ref]         refs/pull/1004/head -> origin/pr/1004

 * [new ref]         refs/pull/1009/head -> origin/pr/1009

...

To check out a particular pull request:

$ git checkout pr/999

Branch pr/999 set up to track remote branch pr/999 from origin.

Switched to a new branch 'pr/999'

we use spf13/cobra for cli's, not urfave

Automatic Changelogs

Changelogs

Requirements

• All changes to the code come through a PR (eg. using GitFlow)
• All PRs generate or modify at least one entry in the changelog
• All entries in the changelog link to all issues addressed by the PRs that generated/modified them
• Compiling changelog should be easy and automatic and intermittent to avoid merge conflict

Problem

The obvious but naiive solution is to update the CHANGELOG.md with every PR. This has problems:

• constant merge conflicts
• changelog entries get added under the wrong version

Ideal

• Every commit is a single full changelog entry
  • eg. "break [x/auth] NewHandler takes a WetKeeper"

But this is probably too extreme of a discipline.

Proposal 1 - Use Commit History

• approximate the ideal
• commits that include #changelog:<type>:<issue> will be included in the changelog
• <type> is one of break, feat, improve, fix
• <issue> is the issue number, or hyphen separated list of them
• every PR includes at least one commit with #changelog:<type>:<issue>
• use a tool to get all commits from all PRs merged since last release and output the full changelog in the form of CHANGELOG.md

Pros

• Approximates the ideal - can work towards it by becoming better committers

Cons

• Complex to support multiple changelog entries in a single commit
• Requires re-writing commit history to fix changelog data

Proposal 2 - Use a Directory of Unique Files

• keep a .changelog directory
• every PR must generate or modify a file in .changelog
• files in .changelog have the same structures as changelog.md
• files in .changelog should be named <author>/<issue> where <author> is the author of the PR and <issue> is the issue number being addressed.
  • If there are multiple issues, list them separated by hyphen. Eg. bucky/1546-1332-1102
• use a tool to merge all files in .changelog and then delete them all

Pros

• easy to make fixes and updates for the same entry by editing a file

Cons

• need to maintain new directory and have extra tmp files in the repo

Tool

• Call it clg for change-log-generator
• clg <version> will output a changelog formatted like CHANGELOG.md

Enforcement

Both proposals can be enforced through rules in testing

Notes

• Use git log 06216c1f^..06216c1f --oneline to print all commits involved in the merge commit `06216c1f
• Use git log --oneline --merges --first-parent <version>..HEAD to get the list of all merge commits to the given branch since

Summary

I'm leaning towards using files. While commit history is the ideal, its hard to realize the ideal and its much more difficult to fix/update the changelog data since they're contained in commits. Using files makes it easiest to find the data and to continuously update it in an auditable way if need be

general repo direction / refactor

• the readme is mostly go advice/tips/standards - these should be pulled into a go_advanced.md that documents standards like cosmos/ethermint-archive#226
• in general, I think this repo should be a recipe book for standardizing repos and the process of merging code in to them. Right now, our ever growing number of repos have various duplicate files & processes.

What are the benefits of a types package?

Personally I feel very strongly about types and would go as far as call it an anti-pattern. Happy to go into detail why, but before curious to hear good arguments for the advocate pratice.

Reference in the tendermint codebase: https://github.com/tendermint/tendermint/tree/master/types

rename PRs from WIP before merging

WIP ends up in commits :(

Add discussion on CodeCov in github branch section

see the conversations in this PR: #53 (review)

new name for make get_vendor_deps

I noticed that it's pretty annoying to type these words, I also noticed that there doesn't seem to be a whole lot of standardization of using the keywords get_vendor_deps - I think this is why back we were using glide I (and many others) would type glide install as opposed to make get_vendor_deps <- it's just much easier to type without underscores... I notice now people are using dep ensure (@ebuchman) sometimes instead now too... so yeah I propose we use something new and fast to type such as:

make dep
make deps
make ensure
make ensuredeps

These can even be in complement to make get_vendor_deps if we really want to maintain that

handling funcs that return errors that are always nil

See tendermint/tendermint#715

Conclusion was to use some PanicOnError func so if it ever changes, we catch it ...

document/standardize how we use hex

• from tendermint/tmlibs#31

tools/Makefile standard

Isolated the tools such that we only need to install glide, then everything else happens from within the tools dir. The binaries are installed in tools/bin after glide install from within the tools dir.

The point is to only require github.com/tendermint/glide, which we control, which becomes the entrypoint to download all other dependencies (including tools).

We can't do it in the project root dir because glide can't deal with multiple glide.yaml files, and there's no way to tell glide to only install the testImports (and there are no "toolImports"). So the only alternative is to put it all in the project root's glide.yaml, but then we end up calling glide install once to download all the vendor deps, and then later we'd end up doing it again, as we like to rm ./vendor upon gen_vendor_deps. So, this is not only more modular (e.g. could be handled as a github submodule perhaps for all projects, and we can put common Makefiles in there), but also more efficient to have two dirs and two glide.yaml files.

cosmos/cosmos-sdk#331

Proposal 1

The proposal is to make this the standard for all of our projects that require tools besides glide. In addition,

• We shouldn't use Mastermind glide, but Tendermint's fork of glide.
• If we need to install gox (@melekes says it doesn't really help), we should install it the same way.

Proposal 2

The next person to touch any Makefile should convert over to using this template by copying over the test dir from github.com/cosmos/cosmos-sdk, and also make the Makefile structure more or less conform to the Makefile standard.

Reorganize Me

This repo is beginning to become a bit of a dump. Discussion should be consolidated and re-organized. Notably I'm not sure why we have so many subdirectories... within go/coding_standards.md there are headers:
"github" <-nothing to do with go
"non-golang code" <- nothing to do with go

how to handle the difference in versioning a lib and its various binaries

from https://github.com/tendermint/basecoin/pull/223 :

So far our standard is to keep versioning information in version pkg. Should we have some standard for keeping the versions of binaries in there too? How do we distinguish binary versions from lib version?

IMO we should have the version/version.go used by a package and all its binaries (i.e. same version). the Makefile can append a commit if we want to get fancy

Add a scripts folder

Test_coverage.sh is a script, not really a description file.. we should think about how to reorg. this repo and possibly export some of these docs to other repos

document which versions of what is meant to be compatible with which version of something else

all: add a .git precommit hook

So the Go project has a precommit hook for Git that checks if all the files are go-fmt'd at
https://github.com/golang/go/blob/964639cc338db650ccadeafb7424bc8ebb2c0f6c/misc/git/pre-commit#L1-L26

I think this could be a start for always ensuring that our source code is go-fmt'd and then later on we can add other rules and tools.

md to rst

https://github.com/kgryte/markdown-to-restructuredtext

consistency for where spec is location in a repo

tendermint: docs/new-spec
abci: specification.rst
cosmos-sdk: docs/spec

These need to be all the same location

project shelldown

Purpose: ensure all docs / tutorials / guides / explainers / hello-world examples are tested.

I plan to begin this once the docs have advanced a bunch more and the repos are a bit more organized, probably around the end of the month.

all: Apache2.0 LICENSE conformation to source code file headers

So I am currently working on adding a doc.go file to go-crypto, and I just noticed that we use the Apache2.0 License in there.

According to the terms of the Apache2.0 license, every source code file SHOULD contain the short header, so that in case code gets intentionally "misused" it is still well protected. Please see http://www.apache.org/dev/apply-license.html#new.

This is what a sample file will look like if we decide to add the short license in all our source files

// Copyright 2017 Tendermint. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package crypto_test

import (
	"fmt"

	"github.com/tendermint/go-crypto"
)

func Example_Sha256() {
	sum := crypto.Sha256([]byte("This is Tendermint"))
	fmt.Printf("%x\n", sum)
	// Output:
	// f91afb642f3d1c87c17eb01aae5cb65c242dfdbe7cf1066cc260f4ce5d33b94e
}

func Example_Ripemd160() {
	sum := crypto.Ripemd160([]byte("This is Tendermint"))
	fmt.Printf("%x\n", sum)
	// Output:
	// 051e22663e8f0fd2f2302f1210f954adff009005
}

create dowloads page for the docs

because tendermint.com/downloads is outdated & the binaries/codebases have been somewhat re-organized. Each ethermint, tendermint, and cosmos-sdk should (I think) have their own download/install page.

implement codeclimate's FIXME engine

ref: #33
and maybe use their linting too? since metalinter isn't w/o issue.

use milestones for issue/project management

Add a proposal-accepted tag

Right now, there isn't a good way to discern via our tagging standards between a proposal we're debating on, and a proposal we've accepted and want to implement. I propose that we add a proposal-accepted tag, and that we remove the proposal tag once its clear that we want to go ahead with a proposal. Golang has one and I think it makes a lot of sense: https://github.com/golang/go/issues?q=is%3Aopen+is%3Aissue+label%3AProposal-Accepted

Prefix issues with what module they affect

The idea is that alot of our issues hint at what module they are affecting already. (This is part of what makes the issue names meaningful) However, the module being affected is placed in various locations on the message, which makes it hard to see at a glance / harder to visually sort through. It would become easier if all of the issues put the module they affected at the beginning instead of various places. So for example:
The following issue:
"Add command to gaiad to show owner_address of a validator"
Would become:
"gaiad: add command to show owner_address of a validator"

I think this is going to be increasingly important now that tendermint is a monorepo. Golang has all issues from all of their extension repos under the same repo, at its easily searchable due to this: https://github.com/golang/go/issues

For issues that users create, we can just rename them, to make the title more reflective of what the underlying issue is. (This is a frequent occurence for golang's issue for example)

Write a specification for our specifications

Ie. cosmos/cosmos-sdk#1268 (comment)

We have lots of specs now, we need to consolidate so they all feel like a common narrative and same style etc.

TODO, BUG, XXX and other warnings should be tracked in issues

we should also consider implementing codeclimate's fixme engine to reduce these occurences in the codebase

Pointers

I feel like one of the least-clear things about Go is when to use pointers vs values. Might be good to explore that.

all the read the docs integrations need a "Edit me HERE" button on each page

that links to the file in the github repo

linting: use a config file instead of metalinter_test

Describe branching and merging process in documentation

• create branches on the repo itself not on a fork to allow other maintainers to rebase your code
• always squash merge into develop from a feature branch
• only use unstable if you need to integrate multiple feature branches before merging into develop
  • unstable should only live for the duration of the merge
• a merge into develop should comprise 1 commit with a properly formatted git title and message

less bash scripts

we have too many and they cause too many problems, as Greg has enlightened me

update readme to clarify how to get PRs merged

from https://github.com/tendermint/internal/issues/19

no anonymous imports!

yes:

import (
    "github.com/dot/package/unique"
)

var ordinary *unique.Whatever

no:

import (
    . "github.com/dot/package/unique"
)

var ordinary *Whatever

all: add a doc.go file for high level intro and dive for packages

Raised offline, let's add a doc.go file to each package whenever we can, to share some high level intro into our packages. I highly believe this will improve developer productivity.

I got bit by the lack of context/familiarity with the libraries in tendermint/go-crypto#35 which supersedes tendermint/go-crypto#32 and tendermint/go-crypto#31, in which I spent a bit of time extracting the code from the libraries yet actually it only required invoking GenEd25519Key().PubKey()

My offline exhibit was:
If so, we can do such a thing as https://github.com/golang/go/blob/0202aa8ec79bf700732812624eedcec3e6e300ad/src/net/http/doc.go#L5-L107 which then gives the Overview at https://golang.org/pkg/net/http/
which gives

https://godoc.org/github.com/golang/go/src/net/http

Add boilerplate code here?

• basecoin plugin
• tendermint plugin
• table driven test
• bash tests
  • init tendermint/basecoin + use shunit2 examples

repos should have an CODEOWNERS file

in .github

`make check` should fail if minimum versions aren't met

putting issue here since it should apply to all repos

original issue: cosmos/cosmos-sdk#463