package X_test forces you to do testing through the public interface. Most of our packages dont use it. When should they ?

we have too many and they cause too many problems, as Greg has enlightened me

see the conversations in this PR: #53 (review)

putting issue here since it should apply to all repos

original issue: cosmos/cosmos-sdk#463

I noticed that it's pretty annoying to type these words, I also noticed that there doesn't seem to be a whole lot of standardization of using the keywords get_vendor_deps - I think this is why back we were using glide I (and many others) would type glide install as opposed to make get_vendor_deps <- it's just much easier to type without underscores... I notice now people are using dep ensure (@ebuchman) sometimes instead now too... so yeah I propose we use something new and fast to type such as:

make dep
make deps
make ensure
make ensuredeps

These can even be in complement to make get_vendor_deps if we really want to maintain that

ref: #33
and maybe use their linting too? since metalinter isn't w/o issue.

See tendermint/tendermint#715

Conclusion was to use some PanicOnError func so if it ever changes, we catch it ...

So the Go project has a precommit hook for Git that checks if all the files are go-fmt'd at
https://github.com/golang/go/blob/964639cc338db650ccadeafb7424bc8ebb2c0f6c/misc/git/pre-commit#L1-L26

I think this could be a start for always ensuring that our source code is go-fmt'd and then later on we can add other rules and tools.

Raised offline, let's add a doc.go file to each package whenever we can, to share some high level intro into our packages. I highly believe this will improve developer productivity.

I got bit by the lack of context/familiarity with the libraries in tendermint/go-crypto#35 which supersedes tendermint/go-crypto#32 and tendermint/go-crypto#31, in which I spent a bit of time extracting the code from the libraries yet actually it only required invoking GenEd25519Key().PubKey()

My offline exhibit was:
If so, we can do such a thing as https://github.com/golang/go/blob/0202aa8ec79bf700732812624eedcec3e6e300ad/src/net/http/doc.go#L5-L107 which then gives the Overview at https://golang.org/pkg/net/http/
which gives

https://godoc.org/github.com/golang/go/src/net/http

So I am currently working on adding a doc.go file to go-crypto, and I just noticed that we use the Apache2.0 License in there.

According to the terms of the Apache2.0 license, every source code file SHOULD contain the short header, so that in case code gets intentionally "misused" it is still well protected. Please see http://www.apache.org/dev/apply-license.html#new.

This is what a sample file will look like if we decide to add the short license in all our source files

// Copyright 2017 Tendermint. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package crypto_test

import (
	"fmt"

	"github.com/tendermint/go-crypto"
)

func Example_Sha256() {
	sum := crypto.Sha256([]byte("This is Tendermint"))
	fmt.Printf("%x\n", sum)
	// Output:
	// f91afb642f3d1c87c17eb01aae5cb65c242dfdbe7cf1066cc260f4ce5d33b94e
}

func Example_Ripemd160() {
	sum := crypto.Ripemd160([]byte("This is Tendermint"))
	fmt.Printf("%x\n", sum)
	// Output:
	// 051e22663e8f0fd2f2302f1210f954adff009005
}

e.g.

--option strings    Genesis option in the format <app>/<option>/<value> (default [stake/allowed_bond_denom/fermion])
--static            use a static private validator

lower case or Upper case?

The idea is that alot of our issues hint at what module they are affecting already. (This is part of what makes the issue names meaningful) However, the module being affected is placed in various locations on the message, which makes it hard to see at a glance / harder to visually sort through. It would become easier if all of the issues put the module they affected at the beginning instead of various places. So for example:
The following issue:
"Add command to gaiad to show owner_address of a validator"
Would become:
"gaiad: add command to show owner_address of a validator"

I think this is going to be increasingly important now that tendermint is a monorepo. Golang has all issues from all of their extension repos under the same repo, at its easily searchable due to this: https://github.com/golang/go/issues

For issues that users create, we can just rename them, to make the title more reflective of what the underlying issue is. (This is a frequent occurence for golang's issue for example)

should be applied to all go code

from https://github.com/tendermint/basecoin/pull/223 :

So far our standard is to keep versioning information in version pkg. Should we have some standard for keeping the versions of binaries in there too? How do we distinguish binary versions from lib version?

IMO we should have the version/version.go used by a package and all its binaries (i.e. same version). the Makefile can append a commit if we want to get fancy

https://github.com/kgryte/markdown-to-restructuredtext

yes:

import (
    "github.com/dot/package/unique"
)

var ordinary *unique.Whatever

no:

import (
    . "github.com/dot/package/unique"
)

var ordinary *Whatever

Right now, there isn't a good way to discern via our tagging standards between a proposal we're debating on, and a proposal we've accepted and want to implement. I propose that we add a proposal-accepted tag, and that we remove the proposal tag once its clear that we want to go ahead with a proposal. Golang has one and I think it makes a lot of sense: https://github.com/golang/go/issues?q=is%3Aopen+is%3Aissue+label%3AProposal-Accepted

A "distribution" event is a singular event with high ramifications (e.g. distributing a hacked binary will affect many people), we might as well take extra precautions while releasing a dist.

We could make the dist, signing, and releasing process automated, and the obvious attack is to hack the signing server or the code that handles the signing.

It's more secure to have many signers, for that mitigates the risk of getting the server hacked. But it doesn't mitigate the risk of getting the dist-building code itself hacked.

A partial solution is to not specify/automate certain things which could be fixed manually. For example, the dist script could choose not to specify a docker image to build under, but rather inherit the environment from the caller.

So the dist-builders are required to find an environment to build in, whether that be a Docker container or something else. This manual setup acts as a kind of fuzzer. There are many configurations that should lead to the same build, such as using Golang version 1.9.x

If we find a discrepancy, it might be because we found a vulnerability, so instead of getting annoyed, we should be thankful and learn what caused the unintended non-determinism, and isolate it, but only it.

I should figure out a way to propose things without using the word "should". I could say "could"...

Changelogs

Requirements

• All changes to the code come through a PR (eg. using GitFlow)
• All PRs generate or modify at least one entry in the changelog
• All entries in the changelog link to all issues addressed by the PRs that generated/modified them
• Compiling changelog should be easy and automatic and intermittent to avoid merge conflict

Problem

The obvious but naiive solution is to update the CHANGELOG.md with every PR. This has problems:

• constant merge conflicts
• changelog entries get added under the wrong version

Ideal

• Every commit is a single full changelog entry
  • eg. "break [x/auth] NewHandler takes a WetKeeper"

But this is probably too extreme of a discipline.

Proposal 1 - Use Commit History

• approximate the ideal
• commits that include #changelog:<type>:<issue> will be included in the changelog
• <type> is one of break, feat, improve, fix
• <issue> is the issue number, or hyphen separated list of them
• every PR includes at least one commit with #changelog:<type>:<issue>
• use a tool to get all commits from all PRs merged since last release and output the full changelog in the form of CHANGELOG.md

Pros

• Approximates the ideal - can work towards it by becoming better committers

Cons

• Complex to support multiple changelog entries in a single commit
• Requires re-writing commit history to fix changelog data

Proposal 2 - Use a Directory of Unique Files

• keep a .changelog directory
• every PR must generate or modify a file in .changelog
• files in .changelog have the same structures as changelog.md
• files in .changelog should be named <author>/<issue> where <author> is the author of the PR and <issue> is the issue number being addressed.
  • If there are multiple issues, list them separated by hyphen. Eg. bucky/1546-1332-1102
• use a tool to merge all files in .changelog and then delete them all

Pros

• easy to make fixes and updates for the same entry by editing a file

Cons

• need to maintain new directory and have extra tmp files in the repo

Tool

• Call it clg for change-log-generator
• clg <version> will output a changelog formatted like CHANGELOG.md

Enforcement

Both proposals can be enforced through rules in testing

Notes

• Use git log 06216c1f^..06216c1f --oneline to print all commits involved in the merge commit `06216c1f
• Use git log --oneline --merges --first-parent <version>..HEAD to get the list of all merge commits to the given branch since

Summary

I'm leaning towards using files. While commit history is the ideal, its hard to realize the ideal and its much more difficult to fix/update the changelog data since they're contained in commits. Using files makes it easiest to find the data and to continuously update it in an auditable way if need be

because tendermint.com/downloads is outdated & the binaries/codebases have been somewhat re-organized. Each ethermint, tendermint, and cosmos-sdk should (I think) have their own download/install page.

in .github

WIP ends up in commits :(

that links to the file in the github repo

I recommend to only include specific tags/releases in your glide.yaml file. That is the only way that you can avoid having the old builds break when someone force pushes to your dependency.

If you are interested in a real world example, please call me :-)

eg. IsDirEmpty (tendermint/tmlibs#34 (comment))

Personally I feel very strongly about types and would go as far as call it an anti-pattern. Happy to go into detail why, but before curious to hear good arguments for the advocate pratice.

Reference in the tendermint codebase: https://github.com/tendermint/tendermint/tree/master/types

• see this comment tendermint/tendermint#678 (comment)

• applied also to ethermint, and ... ?

• create branches on the repo itself not on a fork to allow other maintainers to rebase your code
• always squash merge into develop from a feature branch
• only use unstable if you need to integrate multiple feature branches before merging into develop
  • unstable should only live for the duration of the merge
• a merge into develop should comprise 1 commit with a properly formatted git title and message

via @adrianbrink:

Locate the section for your github remote in the .git/config file. It looks like this:

[remote "origin"]

	fetch = +refs/heads/*:refs/remotes/origin/*

	url = [email protected]:joyent/node.git

Now add the line fetch = +refs/pull/*/head:refs/remotes/origin/pr/* to this section. Obviously, change the github url to match your project's URL. It ends up looking like this:

[remote "origin"]

	fetch = +refs/heads/*:refs/remotes/origin/*

	url = [email protected]:joyent/node.git

	fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

Now fetch all the pull requests:

$ git fetch origin

From github.com:joyent/node

 * [new ref]         refs/pull/1000/head -> origin/pr/1000

 * [new ref]         refs/pull/1002/head -> origin/pr/1002

 * [new ref]         refs/pull/1004/head -> origin/pr/1004

 * [new ref]         refs/pull/1009/head -> origin/pr/1009

...

To check out a particular pull request:

$ git checkout pr/999

Branch pr/999 set up to track remote branch pr/999 from origin.

Switched to a new branch 'pr/999'

Test_coverage.sh is a script, not really a description file.. we should think about how to reorg. this repo and possibly export some of these docs to other repos

I feel like one of the least-clear things about Go is when to use pointers vs values. Might be good to explore that.

• of the open-source stack - to orient newsers

started here tendermint/abci#168

Isolated the tools such that we only need to install glide, then everything else happens from within the tools dir. The binaries are installed in tools/bin after glide install from within the tools dir.

The point is to only require github.com/tendermint/glide, which we control, which becomes the entrypoint to download all other dependencies (including tools).

We can't do it in the project root dir because glide can't deal with multiple glide.yaml files, and there's no way to tell glide to only install the testImports (and there are no "toolImports"). So the only alternative is to put it all in the project root's glide.yaml, but then we end up calling glide install once to download all the vendor deps, and then later we'd end up doing it again, as we like to rm ./vendor upon gen_vendor_deps. So, this is not only more modular (e.g. could be handled as a github submodule perhaps for all projects, and we can put common Makefiles in there), but also more efficient to have two dirs and two glide.yaml files.

cosmos/cosmos-sdk#331

Proposal 1

The proposal is to make this the standard for all of our projects that require tools besides glide. In addition,

• We shouldn't use Mastermind glide, but Tendermint's fork of glide.
• If we need to install gox (@melekes says it doesn't really help), we should install it the same way.

Proposal 2

The next person to touch any Makefile should convert over to using this template by copying over the test dir from github.com/cosmos/cosmos-sdk, and also make the Makefile structure more or less conform to the Makefile standard.

• from tendermint/tmlibs#31

from https://github.com/tendermint/internal/issues/19

This repo is beginning to become a bit of a dump. Discussion should be consolidated and re-organized. Notably I'm not sure why we have so many subdirectories... within go/coding_standards.md there are headers:
"github" <-nothing to do with go
"non-golang code" <- nothing to do with go

• the readme is mostly go advice/tips/standards - these should be pulled into a go_advanced.md that documents standards like cosmos/ethermint-archive#226
• in general, I think this repo should be a recipe book for standardizing repos and the process of merging code in to them. Right now, our ever growing number of repos have various duplicate files & processes.

Priority (color #fad2fa)

• bug
• priority? - use to tag what might be priority

Type (color #d2d2fa)

• proposal - feature requests, enhancement, etc
• question
• task
• policy - process or policy issues
• milestone - an issue describing the milestone

Highlight (color #e6fad2)

• feature - something worth featuring in publications
• bounty - person doing the tagging, or representing some bounty program, will pay a bounty for it.
• help wanted - help wanted from anyone
• good first issue - good entry-level bug, to get familiar w/ the codebase

Concerns (color #fafad2)

• p2p
• rpc
• consensus
• ... project specific, go nuts

Meta (color #e6e6e6)

• duplicate
• wontfix
• fixed

Documentation (color #d2fafa)

• docs - need (updates to) documentation
• spec - need (updates to) specification

Dont use

• invalid -> use wontfix
• enhancement -> use proposal
• priority - high -> use priority? or bug or milestones (codeowner)

Purpose: ensure all docs / tutorials / guides / explainers / hello-world examples are tested.

I plan to begin this once the docs have advanced a bunch more and the repos are a bit more organized, probably around the end of the month.

we should also consider implementing codeclimate's fixme engine to reduce these occurences in the codebase

• basecoin plugin
• tendermint plugin
• table driven test
• bash tests
  • init tendermint/basecoin + use shunit2 examples

I would like to propose that we discourage panics within library/packages unless something is totally unrecoverable or if it is a sanity check. I've seen a bunch of panics in our libraries for functions with signatures that could be modified to return an error. IMHO packages should be the glue code that userland code invokes and shouldn't get surprised by; userland code includes CLIs, servers, handlers etc.
What do y'all think?

tendermint: docs/new-spec
abci: specification.rst
cosmos-sdk: docs/spec

These need to be all the same location

Ie. cosmos/cosmos-sdk#1268 (comment)

We have lots of specs now, we need to consolidate so they all feel like a common narrative and same style etc.