tendermint / go-amino

Protobuf3 with Interface support - Designed for blockchains (deterministic, upgradeable, fast, and compact)

Tendermint

UPDATE: TendermintCore featureset is frozen for LTS, see issue #9972
This is the latest stable release used by cosmoshub-4, version 0.34.24
The previous main branch (v0.38.xx) can now be found under "main_backup"

Byzantine-Fault Tolerant State Machine Replication. Or Blockchain, for short.

Branch	Tests	Linting
main

Tendermint Core is a Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine - written in any programming language - and securely replicates it on many machines.

For protocol details, refer to the Tendermint Specification.

For detailed analysis of the consensus protocol, including safety and liveness proofs, read our paper, "The latest gossip on BFT consensus".

Documentation

Complete documentation can be found on the website.

Releases

Please do not depend on main as your production branch. Use releases instead.

Tendermint has been in the production of private and public environments, most notably the blockchains of the Cosmos Network. we haven't released v1.0 yet since we are making breaking changes to the protocol and the APIs. See below for more details about versioning.

In any case, if you intend to run Tendermint in production, we're happy to help. You can contact us over email or join the chat.

More on how releases are conducted can be found here.

Security

To report a security vulnerability, see our bug bounty program. For examples of the kinds of bugs we're looking for, see our security policy.

We also maintain a dedicated mailing list for security updates. We will only ever use this mailing list to notify you of vulnerabilities and fixes in Tendermint Core. You can subscribe here.

Minimum requirements

Requirement	Notes
Go version	Go 1.18 or higher

Install

See the install instructions.

Quick Start

• Single node
• Local cluster using docker-compose
• Remote cluster using Terraform and Ansible

Contributing

Please abide by the Code of Conduct in all interactions.

Before contributing to the project, please take a look at the contributing guidelines and the style guide. You may also find it helpful to read the specifications, and familiarize yourself with our Architectural Decision Records (ADRs) and Request For Comments (RFCs).

Versioning

Semantic Versioning

Tendermint uses Semantic Versioning to determine when and how the version changes. According to SemVer, anything in the public API can change at any time before version 1.0.0

To provide some stability to users of 0.X.X versions of Tendermint, the MINOR version is used to signal breaking changes across Tendermint's API. This API includes all publicly exposed types, functions, and methods in non-internal Go packages as well as the types and methods accessible via the Tendermint RPC interface.

Breaking changes to these public APIs will be documented in the CHANGELOG.

Upgrades

In an effort to avoid accumulating technical debt prior to 1.0.0, we do not guarantee that breaking changes (ie. bumps in the MINOR version) will work with existing Tendermint blockchains. In these cases you will have to start a new blockchain, or write something custom to get the old data into the new chain. However, any bump in the PATCH version should be compatible with existing blockchain histories.

For more information on upgrading, see UPGRADING.md.

Supported Versions

Because we are a small core team, we only ship patch updates, including security updates, to the most recent minor release and the second-most recent minor release. Consequently, we strongly recommend keeping Tendermint up-to-date. Upgrading instructions can be found in UPGRADING.md.

Resources

Libraries

• Cosmos SDK; A framework for building applications in Golang
• Tendermint in Rust
• ABCI Tower

Applications

• Cosmos Hub
• Terra
• Celestia
• Anoma
• Vocdoni

Research

• The latest gossip on BFT consensus
• Master's Thesis on Tendermint
• Original Whitepaper: "Tendermint: Consensus Without Mining"
• Tendermint Core Blog
• Cosmos Blog

Join us!

Tendermint Core is maintained by Interchain GmbH. If you'd like to work full-time on Tendermint Core, we're hiring!

Funding for Tendermint Core development comes primarily from the Interchain Foundation, a Swiss non-profit. The Tendermint trademark is owned by Tendermint Inc., the for-profit entity that also maintains tendermint.com.

go-amino's Issues

Support arbitrary maps

As long as it's a field, and tagged as wire:ordered, or wire:unsafe. Unsafe is faster, ordered is slower but deterministic.

panic: MarshalBinary cannot marshal a nil pointer.

This is from trying to compute the Merkle tree of Commits, where some of them are nil.

--- FAIL: TestBeginBlockAbsentValidators (0.00s)
panic: MarshalBinary cannot marshal a nil pointer. [recovered]
panic: MarshalBinary cannot marshal a nil pointer.

goroutine 26 [running]:
testing.tRunner.func1(0xc42018c960)
/Users/ethanbuchman/goRoot/1.9.4/src/testing/testing.go:711 +0x2d2
panic(0x15a31a0, 0x1825430)
/Users/ethanbuchman/goRoot/1.9.4/src/runtime/panic.go:491 +0x283
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).MarshalBinary(0xc42009a5a0, 0x15fc100, 0x0, 0x1013088, 0x70, 0x1632400, 0x1, 0xc420920000)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/wire.go:26 +0x2b3
github.com/tendermint/tendermint/wire.MarshalBinary(0x15fc100, 0x0, 0x0, 0x0, 0xc420913100, 0x14, 0x20)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/wire/wire.go:21 +0x41
github.com/tendermint/tendermint/types.hasher.Hash(0x15fc100, 0x0, 0xc420913100, 0x14, 0x20)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/types/block.go:519 +0x85
github.com/tendermint/tendermint/types.(*hasher).Hash(0xc4208df4b0, 0xc420913100, 0x14, 0x20)
:1 +0x45
github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/merkle.SimpleHashFromHashers(0xc42091e000, 0x2, 0x2, 0xc4208df4b0, 0x2, 0x2)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/merkle/simple_tree.go:76 +0xaa
github.com/tendermint/tendermint/types.(*Commit).Hash(0xc4208f8480, 0x1603440, 0xc40b4e3501, 0xc420908d50)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/types/block.go:366 +0x185
github.com/tendermint/tendermint/types.(*Block).FillHeader(0xc420096e60)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/types/block.go:75 +0x12e
github.com/tendermint/tendermint/types.MakeBlock(0x2, 0xc4204b2300, 0xa, 0x10, 0xc4208f8480, 0xc4209114a8)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/types/block.go:39 +0x12b
github.com/tendermint/tendermint/state.State.MakeBlock(0x1695ae1, 0xf, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/state/state.go:112 +0x66
github.com/tendermint/tendermint/state.TestBeginBlockAbsentValidators(0xc42018c960)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/state/execution_test.go:77 +0xa1f
testing.tRunner(0xc42018c960, 0x17da430)
/Users/ethanbuchman/goRoot/1.9.4/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
/Users/ethanbuchman/goRoot/1.9.4/src/testing/testing.go:789 +0x2de
exit status 2
FAIL github.com/tendermint/tendermint/state 0.022s

WriteJSON: handle marshaling nil as a top level value

Go's "encoding/json" can successfully marshal a nil value, whether it is typed or not for example https://play.golang.org/p/dZ_N86Onh8 or inlined

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	b, _ := json.Marshal(nil)
	fmt.Printf("%s\n", b)
}

which gives null

However, our package trips out. The question is: should we allow marshaling nil to null? I realize that most of the usages of this package is with non-nil values but I was just trying to provide test coverage for tendermint/types as per tendermint/tendermint#693 and encountered this. Repro

package main

import (
	"bytes"

	"github.com/tendermint/go-wire"
	"github.com/tendermint/tendermint/types"
)

func main() {
	buf := new(bytes.Buffer)
	var n int
	var err error
	wire.WriteJSON((*types.CanonicalJSONOnceHeartbeat)(nil), buf, &n, &err)
}

which gives

panic: reflect: call of reflect.Value.Field on zero Value

goroutine 1 [running]:
reflect.Value.Field(0x0, 0x0, 0x0, 0x0, 0xc42010c000, 0xc42005df60, 0xc42005df68)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:764 +0x12e
github.com/tendermint/go-wire.writeReflectJSON(0x0, 0x0, 0x0, 0x1694d80, 0x149f960, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect.go:970 +0x117b
github.com/tendermint/go-wire.WriteJSON(0x14367e0, 0x0, 0x168b640, 0xc42010c000, 0xc42005df60, 0xc42005df68)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/wire.go:141 +0x121
main.main()
	/Users/emmanuelodeke/Desktop/experiments/tendermint-data/nil-trip.go:14 +0x84
exit status 2

go-wire ReadBinaryBytes can hang sometime

I have no reproduceable case yet, but I have observed 2 or 3 times now that when running test cases that mutate bytes to generate many thousand examples of mutated data, that occassionally the test will hang. I am pretty sure it is due to the ReadBinaryBytes method, as that was the only code in common between the two test cases (unless somehow MutateByteSlice causes an inifite loop).

So, keep your eyes open...

support canonical json marshaling

eg. for transaction signing/checking, so apps don't have to implement it manually.

should be alphabetically ordered, no spaces.

document and enforce that wire.RegisterInterface requires an exported receiver interface

I spent a bunch of time making repros and not figuring out why they were crashing but alas the problem is that wire requires the receiving interface to be exported i.e start with a capital letter.

Exhibit

Failing case

package main

import (
	"bytes"
	"log"

	"github.com/tendermint/go-wire"
)

type recv interface{}
type person struct {
	Name string
	Age  int
}

func init() {
	_ = wire.RegisterInterface(
		struct{ recv }{},
		wire.ConcreteType{&person{}, 0x01},
	)
}

func main() {
	buf := new(bytes.Buffer)
	var n int
	var err error
	wire.WriteBinary(&person{Name: "AM", Age: 27}, buf, &n, &err)
	if err != nil {
		log.Fatalf("encoding: %v", err)
	}
	am := wire.ReadBinary(struct{ recv }{}, buf, 0, &n, &err).(struct{ recv }).recv.(*person)
	if err != nil {
		log.Fatalf("decoding: %v", err)
	}
	log.Printf("Successfully decoded: %v", am)
}

gives

$ go run repro.go 
panic: interface conversion: main.recv is nil, not *main.person

goroutine 1 [running]:
main.main()
	/Users/emmanuelodeke/Desktop/openSrc/bugs/tendermint/go-wire/652/repro.go:31 +0x2ec
exit status 2

Passing case

Just modifying the case of recv to Recv

--- repro.go	2017-09-19 19:25:22.000000000 -0600
+++ other.go	2017-09-19 19:25:37.000000000 -0600
@@ -7,7 +7,7 @@
 	"github.com/tendermint/go-wire"
 )
 
-type recv interface{}
+type Recv interface{}
 type person struct {
 	Name string
 	Age  int
@@ -15,7 +15,7 @@
 
 func init() {
 	_ = wire.RegisterInterface(
-		struct{ recv }{},
+		struct{ Recv }{},
 		wire.ConcreteType{&person{}, 0x01},
 	)
 }
@@ -28,7 +28,7 @@
 	if err != nil {
 		log.Fatalf("encoding: %v", err)
 	}
-	am := wire.ReadBinary(struct{ recv }{}, buf, 0, &n, &err).(struct{ recv }).recv.(*person)
+	am := wire.ReadBinary(struct{ Recv }{}, buf, 0, &n, &err).(struct{ Recv }).Recv.(*person)
 	if err != nil {
 		log.Fatalf("decoding: %v", err)
 	}

And it passes with no crash

2017/09/19 19:26:06 Successfully decoded: &{AM 27}

Prognosis

Most likely we are just a victim of consequence since this package uses the reflect package to traverse through fields, unexported fields will cause panics if we try to introspect the tags in it,
and moreover for registration one HAS to pass in the receiver.

wire.RegisterInterface(
   struct { <RECEIVER_HERE> },
   ....
)

in the cases where we have an unexported type e.g

type a interface{}

it becomes

wire.RegisterInterface(struct{ a },...)

instead of

type A interface{}
wire.RegisterInterface(struct{ A }, ...)

I haven't investigated further as that'd require ripping through all the code in go-wire but for the sake of time, I can reliably reproduce this problem by switching the cases of the embedded type in the struct passed in during wire.RegisterInterface, as provided in the exhibit.

How does JSON work?

Looking at the tests I see the _df and _v fields.

Is this really what we want users to be seeing in their genesis files and over RPC?

Can we get the old format with type and value back ?

Encoding of a large byteslice causes a runtime panic

The following snippet causes a runtime panic (on go version go1.9 darwin/amd64)

n, err := int(0), error(nil)
var x []byte
wire.ReadBinary(x, bytes.NewReader([]byte{8, 127, 255, 255, 255, 255, 255, 255, 255}), 0, &n, &err)

This is because https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/byteslice.go#L28 allocates a buffer based on the result of ReadVarInt, which can be as large as math.MaxInt64 = 9223372036854775807. This is larger than the maximum slice size. (according to this, the heap size on 64bit architectures is 512gb)

The solution that comes to mind is to make slice lengths int32 instead of int64s. This can be done by casting them after ReadVarInt, or by using ReadInt32 for length-prefixed objects. Either way this gives a max slice length of 2^32-1, or 4GB byteslices. Assuming that's a large enough length --- I'm not familiar enough with the rest of tendermint yet to judge.

If we switch to ReadInt32, that will be a protocol change as []byte{42} used to be encoded as {1,1,42} now becomes {0,0,0,1,42}.

ReadSlice: limit the max length for slices before invoking make([]byte, length)

Just found with go-fuzz, we should check that the number of bytes in the slice doesn't overflow.
The offending code below is at

https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/byteslice.go#L28

In the check right above, we should also perform the due diligence to ensure that we report ErrBinaryReadOverflow

https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/byteslice.go#L23-L26

Repro code:

package main

import "github.com/tendermint/tendermint/types"

func main() {
	vs := new(types.ValidatorSet)
	vs.FromBytes([]byte{0x01, 0x01, 0x30, 0x01, 0x06, 0x30, 0x30, 0x30, 0x30, 0x30,0x30})
}

which gives

panic: runtime error: makeslice: len out of range

goroutine 1 [running]:
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.ReadByteSlice(0x1668ba0, 0xc4205488a0, 0x0, 0xc42018fee0, 0xc42018fef0, 0x86, 0xc420056601, 0x1432820)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/byteslice.go:30 +0x227
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.readReflectBinary(0x1471ce0, 0xc42055e080, 0x197, 0x1672200, 0x1471ce0, 0x1402dcf, 0x7, 0x0, 0x1471ce0, 0xc42000c160, ...)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/reflect.go:277 +0x1169
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.readReflectBinary(0x14a4060, 0xc42055e080, 0x199, 0x1672200, 0x14a4060, 0x1407c55, 0xa, 0x0, 0x1426580, 0xc42000c020, ...)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/reflect.go:317 +0xa96
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.readReflectBinary(0x1426580, 0xc420548870, 0x197, 0x1672200, 0x1426580, 0x1407c55, 0xa, 0x0, 0x1426580, 0xc42000c020, ...)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/reflect.go:292 +0xf53
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.readReflectBinary(0x1492be0, 0xc420548870, 0x199, 0x1672200, 0x1492be0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/reflect.go:317 +0xa96
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.ReadBinary(0x14d23c0, 0xc420548870, 0x1668ba0, 0xc4205488a0, 0x0, 0xc42018fee0, 0xc42018fef0, 0xc42018ff00, 0x1012e38)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/wire.go:31 +0x32e
github.com/tendermint/tendermint/types.(*ValidatorSet).FromBytes(0xc420548870, 0xc420564000, 0xb, 0x20b)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/types/validator_set.go:308 +0xd0
main.main()
	/Users/emmanuelodeke/go/src/github.com/tendermint/tendermint/fuzz/types/validatorSet/repro/main.go:22 +0x142
exit status 2

where:

• length value is: 52983525027888
• lmt: 0

sdk2: tests are failing on go1.10beta* with infinite recursion

The tests are failing when run with go1.10beta* with such a heavy stack trace

$ go test
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

runtime stack:
runtime.throw(0x1367987, 0xe)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/panic.go:616 +0x81
runtime.newstack()
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/stack.go:1054 +0x71f
runtime.morestack()
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:480 +0x89

goroutine 21 [running]:
reflect.(*rtype).NumIn(0x12f7140, 0x0)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/type.go:1025 +0x65 fp=0xc440200330 sp=0xc440200328 pc=0x10a1c65
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc4402006e8, 0x2, 0x2, 0xc4402006b8, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:348 +0xeb fp=0xc4402005f0 sp=0xc440200330 pc=0x10a8f6b
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc4402006e8, 0x2, 0x2, 0x1320fc0, 0xc4402007a0, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440200658 sp=0xc4402005f0 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440200728 sp=0xc440200658 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440200808 sp=0xc440200728 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440200848 sp=0xc440200808 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440200878 sp=0xc440200848 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c160, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc4402008a8 sp=0xc440200878 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440200c60, 0x2, 0x2, 0xc440200c30, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440200b68 sp=0xc4402008a8 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440200c60, 0x2, 0x2, 0x1320fc0, 0xc440200d18, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440200bd0 sp=0xc440200b68 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440200ca0 sp=0xc440200bd0 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440200d80 sp=0xc440200ca0 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440200dc0 sp=0xc440200d80 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440200df0 sp=0xc440200dc0 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c140, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440200e20 sp=0xc440200df0 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc4402011d8, 0x2, 0x2, 0xc4402011a8, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc4402010e0 sp=0xc440200e20 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc4402011d8, 0x2, 0x2, 0x1320fc0, 0xc440201290, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440201148 sp=0xc4402010e0 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440201218 sp=0xc440201148 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc4402012f8 sp=0xc440201218 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440201338 sp=0xc4402012f8 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440201368 sp=0xc440201338 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c120, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440201398 sp=0xc440201368 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440201750, 0x2, 0x2, 0xc440201720, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440201658 sp=0xc440201398 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440201750, 0x2, 0x2, 0x1320fc0, 0xc440201808, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc4402016c0 sp=0xc440201658 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440201790 sp=0xc4402016c0 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440201870 sp=0xc440201790 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc4402018b0 sp=0xc440201870 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc4402018e0 sp=0xc4402018b0 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c100, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440201910 sp=0xc4402018e0 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440201cc8, 0x2, 0x2, 0xc440201c98, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440201bd0 sp=0xc440201910 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440201cc8, 0x2, 0x2, 0x1320fc0, 0xc440201d80, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440201c38 sp=0xc440201bd0 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440201d08 sp=0xc440201c38 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440201de8 sp=0xc440201d08 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440201e28 sp=0xc440201de8 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440201e58 sp=0xc440201e28 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c0e0, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440201e88 sp=0xc440201e58 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440202240, 0x2, 0x2, 0xc440202210, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440202148 sp=0xc440201e88 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440202240, 0x2, 0x2, 0x1320fc0, 0xc4402022f8, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc4402021b0 sp=0xc440202148 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440202280 sp=0xc4402021b0 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440202360 sp=0xc440202280 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc4402023a0 sp=0xc440202360 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc4402023d0 sp=0xc4402023a0 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c0c0, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440202400 sp=0xc4402023d0 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc4402027b8, 0x2, 0x2, 0xc440202788, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc4402026c0 sp=0xc440202400 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc4402027b8, 0x2, 0x2, 0x1320fc0, 0xc440202870, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440202728 sp=0xc4402026c0 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc4402027f8 sp=0xc440202728 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc4402028d8 sp=0xc4402027f8 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440202918 sp=0xc4402028d8 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440202948 sp=0xc440202918 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c0a0, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440202978 sp=0xc440202948 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440202d30, 0x2, 0x2, 0xc440202d00, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440202c38 sp=0xc440202978 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440202d30, 0x2, 0x2, 0x1320fc0, 0xc440202de8, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440202ca0 sp=0xc440202c38 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440202d70 sp=0xc440202ca0 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440202e50 sp=0xc440202d70 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440202e90 sp=0xc440202e50 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440202ec0 sp=0xc440202e90 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c080, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440202ef0 sp=0xc440202ec0 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc4402032a8, 0x2, 0x2, 0xc440203278, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc4402031b0 sp=0xc440202ef0 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc4402032a8, 0x2, 0x2, 0x1320fc0, 0xc440203360, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440203218 sp=0xc4402031b0 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc4402032e8 sp=0xc440203218 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc4402033c8 sp=0xc4402032e8 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440203408 sp=0xc4402033c8 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440203438 sp=0xc440203408 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c060, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440203468 sp=0xc440203438 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440203820, 0x2, 0x2, 0xc4402037f0, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440203728 sp=0xc440203468 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440203820, 0x2, 0x2, 0x1320fc0, 0xc4402038d8, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440203790 sp=0xc440203728 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440203860 sp=0xc440203790 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440203940 sp=0xc440203860 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440203980 sp=0xc440203940 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc4402039b0 sp=0xc440203980 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c040, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc4402039e0 sp=0xc4402039b0 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440203d98, 0x2, 0x2, 0xc440203d68, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440203ca0 sp=0xc4402039e0 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440203d98, 0x2, 0x2, 0x1320fc0, 0xc440203e50, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440203d08 sp=0xc440203ca0 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440203dd8 sp=0xc440203d08 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440203eb8 sp=0xc440203dd8 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440203ef8 sp=0xc440203eb8 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440203f28 sp=0xc440203ef8 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c020, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440203f58 sp=0xc440203f28 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440204310, 0x2, 0x2, 0xc4402042e0, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440204218 sp=0xc440203f58 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440204310, 0x2, 0x2, 0x1320fc0, 0xc4402043c8, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440204280 sp=0xc440204218 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440204350 sp=0xc440204280 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440204430 sp=0xc440204350 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440204470 sp=0xc440204430 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc4402044a0 sp=0xc440204470 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc42112c000, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc4402044d0 sp=0xc4402044a0 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440204888, 0x2, 0x2, 0xc440204858, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440204790 sp=0xc4402044d0 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440204888, 0x2, 0x2, 0x1320fc0, 0xc440204940, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc4402047f8 sp=0xc440204790 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc4402048c8 sp=0xc4402047f8 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc4402049a8 sp=0xc4402048c8 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc4402049e8 sp=0xc4402049a8 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440204a18 sp=0xc4402049e8 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc421129fe0, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440204a48 sp=0xc440204a18 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440204e00, 0x2, 0x2, 0xc440204dd0, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440204d08 sp=0xc440204a48 pc=0x10a97d5
reflect.Value.Call(0x12f7140, 0x137aab0, 0x13, 0xc440204e00, 0x2, 0x2, 0x1320fc0, 0xc440204eb8, 0x133c920)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:308 +0xa4 fp=0xc440204d70 sp=0xc440204d08 pc=0x10a8d64
github.com/google/gofuzz.(*Fuzzer).tryCustom(0xc4200a1290, 0x12d9d80, 0xc4201400d0, 0x16, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:296 +0x2c5 fp=0xc440204e40 sp=0xc440204d70 pc=0x12a29f5
github.com/google/gofuzz.(*Fuzzer).doFuzz(0xc4200a1290, 0x12e7440, 0xc4201400d0, 0x197, 0x0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:188 +0xb27 fp=0xc440204f20 sp=0xc440204e40 pc=0x12a2617
github.com/google/gofuzz.Continue.Fuzz(0xc4200a1290, 0xc4200a1350, 0x12d9d80, 0xc4201400d0)
	/Users/emmanuelodeke/go/src/github.com/google/gofuzz/fuzz.go:328 +0x93 fp=0xc440204f60 sp=0xc440204f20 pc=0x12a2d43
github.com/tendermint/go-wire.glob..func1(0xc4201400d0, 0xc4200a1290, 0xc4200a1350)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:410 +0x4b fp=0xc440204f90 sp=0xc440204f60 pc=0x12b2c6b
runtime.call32(0xc4200a1380, 0x137aab0, 0xc421129fc0, 0x1800000018)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/runtime/asm_amd64.s:573 +0x3b fp=0xc440204fc0 sp=0xc440204f90 pc=0x10579eb
reflect.Value.call(0x12f7140, 0x137aab0, 0x13, 0x13650c0, 0x4, 0xc440205378, 0x2, 0x2, 0xc440205348, 0x1358300, ...)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/reflect/value.go:447 +0x955 fp=0xc440205280 sp=0xc440204fc0 pc=0x10a97d5
...additional frames elided...
created by testing.(*T).Run
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:824 +0x2ca

goroutine 1 [chan receive]:
testing.(*T).Run(0xc4201421e0, 0x1369f93, 0x15, 0x137aa98, 0x1076f01)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:825 +0x2eb
testing.runTests.func1(0xc420142000)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:1063 +0x64
testing.tRunner(0xc420142000, 0xc42005fdf8)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:777 +0xc4
testing.runTests(0xc420128480, 0x15344c0, 0x11, 0x11, 0x1012149)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:1061 +0x2bb
testing.(*M).Run(0xc420140000, 0x0)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:978 +0x168
main.main()
	_testmain.go:78 +0x151

goroutine 20 [chan receive]:
testing.(*T).Run(0xc4201422d0, 0x12cd844, 0x10, 0xc420128540, 0x10dc6dd)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:825 +0x2eb
github.com/tendermint/go-wire.TestCodecBinaryStruct(0xc4201421e0)
	/Users/emmanuelodeke/go/src/github.com/tendermint/go-wire/reflect_test.go:230 +0x5f
testing.tRunner(0xc4201421e0, 0x137aa98)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:777 +0xc4
created by testing.(*T).Run
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/testing.go:824 +0x2ca
exit status 2
FAIL	github.com/tendermint/go-wire	2.092s

Panic caused by decoding invalid time.

There's a panic here:
https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/time.go#L23-L25

that can be triggered with malicious user input.

I.e. the following test panics:

func TestTimeToPanic(t *testing.T) {
	var x time.Time
	n, err := int(0), error(nil)
	theTime := ReadBinary(x, bytes.NewReader([]byte{0, 0, 0, 0, 0, 0, 0, 1}), 8, &n, &err)
	fmt.Println(theTime)
}

This is a denial of service vector similar to #25

We should instead do something like t = t / 1000000 * 1000000 or just error out.

EncodeTime: fix comment about cast to int32

Currently in the code on develop we have https://github.com/tendermint/go-wire/blob/dec83f641903b22f039da3974607859715d0377e/encoder.go#L126

saying 'this int64 --> int32 is safe.'

However, that's a little bit misleading because t.NanoSecond() returns an int
and the range of nano-seconds is guaranteed to be [0, 999999999]
as per https://golang.org/pkg/time/#Time.Nanosecond

plus the range of int is the same as the range of int32 i.e, -2147483648 to 2147483647.

Perhaps we meant to make that comment

// this int --> int32 is safe as they have the exact same range plus
// nano-seconds are in the range [0, 999999999] < ((1<<32) - 1) .

I mention this because I was just auditing the code and preparing for a fuzz so the comment initially misled me into thinking this was something to test as int64 --> int32 is a potential truncation that could have interesting results, but on recalling and then looking up that time.Time.Nanosecond returns an int.

reconsider silent encoding of struct without interface registered causing omitted type field

it seems weird to me that when one fails to wire.RegisterInterface in advance, the system still produces an output but it is missing the type byte and has the length varint there instead.

0102010106536E6F6F70790301054461697379

➜  nowriter git:(develop) ✗ vi t.go // comment out RegisterInterface call

➜  nowriter git:(develop) ✗ go run t.go
01020106536E6F6F707901054461697379

➜  nowriter git:(develop) ✗ 

based on Animal example in go-wire main doc page on github with wire.RegisterInterface commented out in the second case.

here's the problem in wire.go: https://github.com/tendermint/go-wire/blob/1019b2ded93cb6251c9836402b8ca358e18ce1bc/reflect.go#L462

Should we write a type byte in that case (such as 0xff) in order to be able to have an easy way to recognize and consistent way to get a type tag even if source code changed and a type was written that was not registered (such as a new type)?

or should we instead introduce a new Option field such as strict to panic in this case? it seems unsafe to have a type field in the data stream that depends on whether or not the programmer has previously registered the type that silently emits the same info with or without the type field in front and no safe way to distinguish.

[2:13]
strict can mean "panic if you encounter any requests to write unregistered interfaces" instead of silently writing them for convenience.

[2:14]
i would think we would want to default to strict mode.

rudi
[2:28 PM]
i wouldn't mind that functionality ... as long as it was a different function. my problem with it is that the presence or absence of a type tag in the output data stream written is determined by whether or not the developer has called RegisterInterface in advance or not. That seems risky to me.

[2:29]
In the event that such an error occurs, it is hard to detect since the next byte could be anything.

Add code generation for go-data adaptors

Every interface we want to use with go-data's json adaptor needs to be embedded in a struct and have some boilerplate methods.

Some examples are here:

https://github.com/tendermint/go-crypto/blob/develop/pub_key.go#L23-L67
https://github.com/tendermint/go-crypto/blob/develop/priv_key.go#L131-L133

https://github.com/tendermint/go-crypto/blob/develop/priv_key.go#L36-L63

This become boilerplate we generate (and then have to possibly maintain) between all usages of the go-data json marshaling. It would be nice to be able to auto-generate this.

There is a nice library to enable this kind of template-base codegen. more docs here

Type Assert data.Bytes on protobuf Marshaler/Unmarshaler

#52 (comment)

Get rid of global Marshal* Register* functions

If you rely on the global methods, any library can register concrete on some structure that you weren't expecting to be concrete.

This is a side-effect of wanting to always include the 4-byte prefix. This allows us to marshal the concrete type, and unmarshal into an interface.

But this feature is incompatible with safety in the context of a globally shared codec. Nobody can safely use wire.Marshal/Unmarshal, so we shouldn't even provide it in the first place.

I think we should also change the behavior so that RegisterConcrete is like taking an option to "always include 4 prefix bytes", which is not the default option. By default, the 4 bytes would not be written unless being encoded as an interface value.

Go-crypto would "always include 4 prefix bytes".

Allow more than 1 TypeByte

I am sure 256 types seems like quite a few implementations, but slowly this is becoming an issue with basecoin aka quark

We define one Tx struct and a TxInner interface that all transactions must implement in every app. This allows us to very easily marshal and unmarshal both json and binary data, handle nested tx transparently, etc. I am very happy with this. However.... we currently have 15 tx types in the core modules, and maybe 5 more in planning... plus at least 10 more in implementing elections. And trying to avoid accidental collisions, rigel and I have been spacing these out a bit. No real problem so far, but as soon as third-parties start writing modules and defining their own tx types (trackomatron, gaia, basecoin-stake, etc.) the number will get larger and will need to coordinate between multiple authors to avoid collisions. Or have modules that cannot both be installed in the same app due to this issue.

I would like to have 2 type bytes for tx, use the first 12 bits for the modules and the last 4 (last hex digit) for the tx in the repo itself. Maybe even consider 3 bytes... but maybe that is too much. Anyway, flexibility in our encoding system would be great.

I understand that go-wire is used in many different places in tendermint core, where this limit is not a problem, and they wish to keep 1 byte prefixes, and we must maintain the same binary encoding.

I would propose a system where wire.RegisterInterface maintains the exact same semantics along with the same 1 byte prefix, but there is another function, eg. wire.RegisterLongerInterface(interface{}, prefix int, ...LongerConcreteTypes ) and LongerConcreteTypes allows []byte instead of byte, and asserts that all types have the same length as defined in the call.

This can create and enhanced TypeInfo, that knows how many bytes it should take off for the comparison (as it is registered per interface), and extends ByteToType and TypeToByte. I would also expand data.Mapper.RegisterImplementation to accept multiple bytes and do the same assertions (all same prefix length) internally.

In short: func (m Mapper) RegisterImplementation(data interface{}, kind string, b byte) Mapper would become func (m Mapper) RegisterImplementation(data interface{}, kind string, b ...byte) Mapper and drop in compatible with any code that uses it.

If all modules now use this mapper instead of directly calling wire.RegisterInterface, I could change the functionality there directly instead of implementing RegisterLongerInterface

use testing/quick for encoding + encode/decode round trips

import "testing/quick"
...
func TestArray(t *testing.T) {                                                                                                           
  propertyTest := func(input []uint8) bool {                                                                                             
    truncatedInputLen := len(input) / 2 * 2                                                                                              
    truncatedInput := input[:truncatedInputLen]                                                                                          
                                                                                                                                         
    pairs := []uint16{}                                                                                                                  
    for i := 0; i+1 < truncatedInputLen; i += 2 {                                                                                        
      pair := uint16(truncatedInput[i]<<8) + uint16(truncatedInput[i+1])                                                                 
      pairs = append(pairs, pair)                                                                                                        
    }                                                                                                                                    
                                                                                                                                         
    encodedBytes := arrayRaw.EncodeUint16Array(pairs)                                                                                    
                                                                                                                                         
    return bytes.Compare(encodedBytes, truncatedInput) == 0                                                                                       
  }                                                                                                                                      
                                                                                                                                         
  assert.Nil(t, quick.Check(propertyTest, nil))                                                                                          
}

Consider 32-bit systems

We probably want this to work on 32-bit systems, and there are some places where we cast to int or uint, which might compromise things.

At some point we should decide if/that we need to support 32-bit and deal with this.

UnmarshalBinary and UnmarshalJSON should not panic on nil deference

The above codec methods should return an error like the stdlib does instead of panicking. I agree this is too distracting for us not to gracefully fail.

https://play.golang.org/p/ltv0I7O_YjS

package main

import (
	"encoding/json"
	"log"
)

func main() {
	var df *int
	if err := json.Unmarshal([]byte("10"), df); err != nil {
		log.Fatal(err)
	}
}

Raised by @veorq offline.

JSONBytesPtr and BinaryBytesPtr

So we don't have to do the struct{ MyInterface }{foo} business.

Support MarshalWire, UnmarshalWire

MarshalWire() and UnmarshalWire() take the following set of method signature formats:

(_ MyObject) MarshalWire() (repr <T>, err error)
(_ MyObject) UnmarshalWire(repr <T>) (err error)

For example,

type Rat struct {
  *big.Rat
}

// RatWire  - Wire Marshalable Rat Struct
type RatWire  struct {
    Numerator   int64 `json:"numerator"`
    Denominator int64 `json:"denominator"`
}

func (r Rat) MarshalJSON() (repr RatWire, err error) {
    ...
}

// UnmarshalJSON - custom implementation of JSON Unmarshal
func (r *Rat) UnmarshalWire(repr RatWire) error {
     ...
}

Benefits: Works for all encoding schemes, e.g. even if we were to support say MarshalProto.

Reading byte array types doesn't respect `lmt`

When a bytearray is read by readReflectBinary
https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/reflect.go#L249-L257

the check to make sure that the length being read doesn't go over lmt is skipped

This check will still eventually happen in ReadBinary https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/wire.go#L39-L41 but it happens outside of readReflectBinary

OTOH it can be argued that since array lengths (unlike slice lengths) are set by the library user, if they try to read a type with a fixed-length bytearray that's longer than lmt they made the mistake themselves.

all: go fmt

I just peeked into this code and noticed un-gofmt'd code that has been in since like 2016. Could we please implement a pre/post commit hook that ensures all *.go files are go fmt'd?

https://github.com/tendermint/go-wire/blob/27be46e25124ddf775e23317a83647ce62a93f6b/expr/expr.go#L2-L766

Document non-extensibility of wire encoded types

eg. #35

fuzz testing for go-wire

We've refurbished the go-wire code from the sdk binary and json codec rewrites. It'd nice for us to fuzz all of them with byte inputs(ie decode) as @jaekwon already started fuzzing with structs(encode then decode) as per https://github.com/tendermint/go-wire/blob/dec83f641903b22f039da3974607859715d0377e/reflect_test.go#L242-L463

support `omitempty`

Test for when UnmarshalBinary in a struct interface field with nil value

We clearly weren't testing, since this fix went unnoticed by tests:

c7801c1

Binary decode bug in decoding registered concrete type into concrete var.

PR incoming.

Why Uint16s

We have functions for writing []uint16 but not anything else.

Do we need that? Maybe we can remove.

Also, the implementation appears buggy: https://github.com/tendermint/go-wire/blob/develop/nowriter/tmdecoding/tm_decoder_pure.go#L158

• should use +2 and needs to slice off the first 4 bytes (the length) before reading the rest

doc.go, provide examples and high level description of go-wire and how to use it

I realize that go-wire is a low-level codec, comparable with https://golang.org/pkg/encoding/gob/
and this of course requires one to register each type and set its identifying ID or type byte.

However, the package is quite undocumented and requires guessing around in order to use it. We could use a doc.go file at the top level and perhaps some examples.

It took me having to spend hours going through all the pieces in PR tendermint/tendermint#648 and also lost time trying to figure out what was wrong with tendermint/tendermint#652 (comment).

I personally don't think that I am a Go newbie, I am familiar with codecs but even that it has taken me a very long time to stitch things up with go-wire, so I can only empathize with the typical beginner, moreover dealing with advanced concepts such as the blockchain to top it all off.

This is a rant, it's just me trying to articulate why a doc.go file and examples are important to ease with development and enable other developers. I believe we could recoup lost time and also reduce the bus factor with the updates requested.

unify go-data json and go-wire json

@ethanfrey commented on Thu Mar 02 2017

go-data json makes for nice apis and handles all the types that go-wire does. however, the json format is designed for readability and apis, rather than determinism. in fact, setting the global data.Encoder will definitely lead to different results in different apps. This is fine for an exchange format, but not for anything that goes into app logic (the parsed json as a struct is deterministic)

Jae mentioned replacing the go-wire json with the go-data json, which I am not completely against, but we must first check where we rely on the determinism of go-wire json. A quick search in the codebase came up with at least 4 places, where go-wire JSON is used to build sign-bytes. Which must be deterministic to reliably validate anything. These methods (at least) will need to find an alternative encoding if we modify the go-wire json style.

This is related to the following issue: #8

https://github.com/tendermint/tendermint/blob/master/types/vote.go#L59-L64
https://github.com/tendermint/tendermint/blob/master/types/proposal.go#L43-L48
https://github.com/tendermint/tendermint/blob/master/types/part_set.go#L76-L78
https://github.com/tendermint/tendermint/blob/master/types/block.go#L408-L415

Also see:
https://github.com/tendermint/tendermint/blob/master/types/canonical_json.go

---

@ethanfrey commented on Fri May 12 2017

Okay, the rpc interface no longer uses go-wire json in tendermint:develop.

We still use this for canonical json generation, which we should revisit as well.

UvarintSize is unnecessarily expensive

I was just auditing the code for fuzzing when I noticed that UvarintSize while trying to find the size of an uint64 actually performs a full binary.PutUvarint encoding and then only extracts the number of bytes written.

We can improve this:
In order to count the number of bytes, we can simply deduct off a byte each time i.e 7 bits off each time so for example to count that we have 3 bytes in 0xFFF, our steps will constitute
while n >= 128 ie 0x80, since byte values are from -128 to 127
a) increment i
b) right shift by 7 bits to clear out the byte length that we just counted
thus making this simple algorithm

i := 1
for n >= 0x80 {
   n >>= 7
   i++
}
return i

Given these benchmarks at https://gist.github.com/odeke-em/1b0bdd33304ab3190e6a09776880dae9#file-uvarintsize_test-go

$ benchstat old.txt new.txt 
name    old time/op    new time/op    delta
Size-4     139ns ± 3%      83ns ± 2%  -40.39%         (p=0.000 n=9+10)

name    old alloc/op   new alloc/op   delta
Size-4    0.00B ±NaN%    0.00B ±NaN%     ~     (all samples are equal)

name    old allocs/op  new allocs/op  delta
Size-4     0.00 ±NaN%     0.00 ±NaN%     ~     (all samples are equal)

and we gain a ~40% improvement in runtime, plus our function won't have much to clean up on the stack

Make go-data json setup comprehensible

codecs: implement custom byte types Vs only Bytes and defining global codecs

@jaekwon asked me to post up some comments that I made on PR tendermint/tmlibs#102
about defining codecs instead of just defining custom byte types.
TL;DR: instead of defining multiple codecs and then using init in code that might need to encode/decode Bytes differently, let's define derivative Bytes each with their custom Marshal methods then freely use those types e.g

• Bytes
• Base64Bytes
• HexBytes
• RawBase64Bytes
  etc

Long wind

And here are the comments verbatim from tendermint/tmlibs#102 (comment)

I definitely could be wrong but here are some thoughts:
I am not convinced that we should have a default codec and then use it in here magically on behalf of the user.
Why not let the user call say HexCodec.UnmarshalJSON(data)? What happens when we have two different programs that each import this package but might want to differently encode Byte?

For example if I have a struct with the address stored in hex but the digest, stored in base64

type Data struct {
    Address cmn.Byte `json:"addr"` // <- to be stored in hex
    PassphraseDigest cmn.Byte `json:"pass_addr"` // <-- to be stored in Base64
}

then oops in order to successfully encode/decode that struct, I will have to implement a custom MarshalJSON, UnmarshalJSON that has to set the codec after each field has been traversed or invoke a separate codec for each field. Moreover this most likely would be done concurrently encoding and decoding other structs. It also would require the user to be an advanced Go user to successfully use the data types thus a higher mental load on the user. For example, this is what I think it will take for a user to successfully marshal the struct defined above that has fields that both are of type cmn.Bytes but they want the representations to be different on the wire:

func (d *Data) MarshalJSON() ([]byte, error) {
   var err error
   repr := make(map[string]interface{})
   if repr["addr"], err = cmn.HexCodec.MarshalJSON(repr.Address); err != nil {
      return nil, err
   }
   if repr["pass_digest"], err = cmn.Base64Codec.MarshalJSON(repr.PassphraseDigest); err != nil {
      return nil, err
   }
   // Then for the other fields of the Data struct
   overallMap := make(map[string]interface{})
   overallBlob, err := json.Marshal(d)
   if err != nil {
      return nil, err
   }
   if err := json.Unmarshal(overallBlob, &overallMap); err != nil {
      return nil, err
   }
   // Now overwrite the keys of the custom base64 and hex keys encoded differently together
   for k, v := range repr {
       overallMap[k] = v
   }
   return json.Marshal(overallMap)
}

and notice how that requires advanced knowledge of the encoding/json package but also that the complexity increases with every single custom field added plus it'll do double work in the step of encoding the non-cmn.Bytes fields

Proposed change

I defined separate types with different codecs, I now only need to define the struct as:

type Data struct {
   Address cmn.HexBytes `json:"addr"`
   PassphraseDigest cmn.Base64Bytes `json:"pass_digest"`
}

Instead of implementing *Coder, perhaps we couldimplement different *Byte types, each with its respective Marshal and String methods. This way the user doesn't have to first set a global codec everytime, plus they now have better control in structs.

type JSONBytes []byte
var _ json.Codec = (*JSONBytes)(nil)
var _ fmt.Stringer = (*JSONBytes)(nil)
type HexBytes baseBytes
var _ json.Codec = (*HexBytes)(nil)
var _ fmt.Stringer = (*HexBytes)(nil)
type Base64Bytes baseBytes
var _ json.Codec = (*Base64Bytes)(nil)
var _ fmt.Stringer = (*Base64Bytes)(nil)

// Implement the respective *arshal* and String methods for each type

How to unmarshal from a reader?

Do we really just need to read everything manually one byte at a time or whatever?

Otherwise there's no way to know how long a buffer to create and read into to unmarshal from.

Eg. in Tendermint p2p, we have to read msgPacket. We'll have to

read 1 byte - channel ID
read 1 byte - EOF
read length of byte slice
read byte slice

What is "wire"?

So I'm trying to make a Python3 compatible example for the tmsp. And came across this error[1] in the current Python 2 version. But reading through this repo I'm trying to understand what is the goal of making a new protocol. This seems really close to BSON but the benefit of BSON is that is supports hash mapping.

https://en.wikipedia.org/wiki/BSON

[1]

python ./tmsp/wire.py
Traceback (most recent call last):
  File "./tmsp/wire.py", line 94, in <module>
    ds = map(decode_big_endian, bs,ss)
  File "./tmsp/wire.py", line 33, in decode_big_endian
    firstByte = reader.read(1)[0]
AttributeError: 'bytearray' object has no attribute 'read'

sdk2: fix invalid and expensive byte trim code

I was just reviewing the code to see why our tests take 67seconds to run and noticed that
throughout in the code we have patterns of code such as
https://github.com/tendermint/go-wire/blob/c7801c1586f51bb28028cd420c599516d7ac9c36/reflect.go#L809-L811

and
https://github.com/tendermint/go-wire/blob/c7801c1586f51bb28028cd420c599516d7ac9c36/reflect.go#L814-L816

the above pattern is both invalid and slow, it'll runtime panic if the entire byte slice has 0x00 but also it resizes the slice every single time.

We can generate faster code and correct code by doing

func trimNullBytesPrefix(bz []byte) {
        i := 0
        for i < len(bz) {
                if bz[i] != 0x00 {
                        break
                }
                i++
        }
        if i >= 1 {
                bz = bz[i:]
        }
}

sdk: run vet on codec.go

While working on a feature for sdk, I was running tests and encountered this

# github.com/tendermint/go-wire
./codec.go:215: Errorf format %v reads arg #2, but call has only 1 arg
./codec.go:406: Sprintf format %v reads arg #3, but call has only 2 args
FAIL	github.com/tendermint/go-wire [build failed]

/cc @jaekwon.

make LengthPrefixedEncoder

same as UnlengthPrefixEncoder for block writing array primitives but prepending some kind of integer type in advance to say how many elements

design decision: do we use the same fixed length int everywhere? like 32 bits maybe? or do we use varint everywhere? or do we need multiple interface choices here?

allocation size out of range

From Tendermint types testing/fuzzing:

--- FAIL: TestTxProofUnchangable (2.08s)
panic: runtime: allocation size out of range [recovered]
panic: runtime: allocation size out of range

goroutine 51 [running]:
testing.tRunner.func1(0xc4208d2870)
/Users/ethanbuchman/goRoot/1.9.4/src/testing/testing.go:711 +0x2d2
panic(0x156a980, 0x179e970)
/Users/ethanbuchman/goRoot/1.9.4/src/runtime/panic.go:491 +0x283
reflect.unsafe_NewArray(0x152ea60, 0x1005530cfb9, 0x152ea60)
/Users/ethanbuchman/goRoot/1.9.4/src/runtime/malloc.go:858 +0x35
reflect.MakeSlice(0x1a44060, 0x152a860, 0x1005530cfb9, 0x1005530cfb9, 0x6, 0x0, 0x0)
/Users/ethanbuchman/goRoot/1.9.4/src/reflect/value.go:2067 +0xc7
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).decodeReflectBinarySlice(0xc420064420, 0xc420b89654, 0x6a, 0x6a, 0xc420f6a270, 0x152a860, 0xc420b79a80, 0x197, 0x14f96ee, 0x5, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:448 +0x3f2
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec)._decodeReflectBinary(0xc420064420, 0xc420b8964e, 0x70, 0x70, 0xc420f6a270, 0x152a860, 0xc420b79a80, 0x197, 0x14f96ee, 0x5, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:120 +0x1db8
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).decodeReflectBinary(0xc420064420, 0xc420b8964e, 0x70, 0x70, 0xc420f6a270, 0x
152a860, 0xc420b79a80, 0x197, 0x14f96ee, 0x5, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:42 +0x375
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).decodeReflectBinaryStruct(0xc420064420, 0xc420b8964e, 0x70, 0x70, 0xc420f6a1a0, 0x1580a80, 0xc420b79a80, 0x199, 0x14de493, 0x5, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:493 +0x278
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec)._decodeReflectBinary(0xc420064420, 0xc420b8964e, 0x70, 0x70, 0xc420f6a1a0, 0x1580a80, 0xc420b79a80, 0x199, 0x14de493, 0x5, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:125 +0x1b95
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).decodeReflectBinary(0xc420064420, 0xc420b8964e, 0x70, 0x70, 0xc420f6a1a0, 0x1580a80, 0xc420b79a80, 0x199, 0x14de493, 0x5, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:42 +0x375
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).decodeReflectBinaryStruct(0xc420064420, 0xc420b8964e, 0x70, 0x70, 0xc420f6a0d0, 0x15d1d60, 0xc420b79a40, 0x199, 0x0, 0x0, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:493 +0x278
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec)._decodeReflectBinary(0xc420064420, 0xc420b895e0, 0xde, 0xde, 0xc420f6a0d0, 0x15d1d60, 0xc420b79a40, 0x199, 0x0, 0x0, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:125 +0x1b95
github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire.(*Codec).decodeReflectBinary(0xc420064420, 0xc420b895e0, 0xde, 0xde, 0xc420f6a0d0, 0x15d1d60, 0xc420b79a40, 0x199, 0x0, 0x0, ...)
/Users/ethanbuchman/programming/goApps/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-wire/binary-decode.go:42 +0x375

Cannot json serialize map

JSON encoding handles primative types, structs, arrays, and anything with a registered handler, but it does not support maps. Maybe this is a design decision?
https://github.com/tendermint/go-wire/blob/develop/json-encode.go#L16-L106

We need to use wire.Un/MarshalJSON() to serialize and deserialize all rpc endpoints and events, so they can handle the interfaces well. However, dump_consensus_state requires that we send a map over rpc:
https://github.com/tendermint/tendermint/blob/breaking/wire-sdk2/rpc/core/types/responses.go#L105-L109

The key p2p.ID is an alias for string, so this is a standard json object. If we do not require determinism for wire.MarshalJSON, this should be easy. If we do, then maybe we can encode all maps as "CanonicalJSON" and this helps with that code in tendermint core as well.

Panic when feeding invalid data

In my tests, I marshal a valid proof to bytes, then fuzz it with MutateBytes, and get quite a few panics (which I had to catch). It would be nice to make go-wire a bit more robust:

Seems if no limit is passed in (ReadBinaryBytes), this length can overflow.
https://github.com/tendermint/go-wire/blob/master/byteslice.go#L28

runtime error: makeslice: len out of range
github.com/tendermint/light-client/proofs.AppProver.Unmarshal.func1
    /Users/ethan/golang/src/github.com/tendermint/light-client/proofs/app.go:64
runtime.call32
    /usr/local/go/src/runtime/asm_amd64.s:479
runtime.gopanic
    /usr/local/go/src/runtime/panic.go:458
runtime.makeslice
    /usr/local/go/src/runtime/slice.go:49
github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire.ReadByteSlice
    /Users/ethan/golang/src/github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire/byteslice.go:28
github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire.readReflectBinary
    /Users/ethan/golang/src/github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire/reflect.go:277
github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire.readReflectBinary
    /Users/ethan/golang/src/github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire/reflect.go:317
github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire.ReadBinaryPtr
    /Users/ethan/golang/src/github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire/wire.go:48
github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire.ReadBinaryBytes
    /Users/ethan/golang/src/github.com/tendermint/light-client/vendor/github.com/tendermint/go-wire/util.go:25
github.com/tendermint/light-client/proofs.AppProver.Unmarshal
    /Users/ethan/golang/src/github.com/tendermint/light-client/proofs/app.go:72
github.com/tendermint/light-client/proofs.(*AppProver).Unmarshal

For some strange reason the same fuzz code doesn't cause a panic with this struct:

type TxProof struct {
	Height uint64
	Index  int
	Txs    types.Txs
}

codec.go is dead code

might as well rm it.

Test tendermint/binary binary+json encoding+decoding of bytearrays and other-arrays

Fixed length arrays! We need to test them.

make raw UnlengthPrefixedArrayEncoder

just array versions of primitives for efficiency and convenience and cleanliness in advance because they are so easy. 8x: 8,16,32,64 and int/uint

Changed binary format of ABCI messages since tendermint 0.16

Hi,

we are trying to upgrade jabci to 0.16 (https://github.com/jTendermint/jabci) and we are totally lost on the following behavior:

Up until tendermint 15, the initial message when tendermint connected to the ABCI APP was
0x01,0x13,0x22,0x11,0x0a,0x0f,0x30,0x2e,0x31,0x35,0x2e,0x30,0x2d,0x61,0x35,0x62,0x37,0x30,0x33,0x34,0x64,0x01,0x02,0x1a,0x00
Which translates to varint size 0x01 and therefor message size 0x13 which we could then parse to the appropriate protobuf type.
Now, the initial messages has a completely different prefix:
0x26,0x22,0x11,0x0a,0x0f,0x30,0x2e,0x31,0x36,0x2e,0x30,0x2d,0x63,0x38,0x61,0x32,0x62,0x64,0x66,0x37,0x04,0x1a,0x00

Interestingly enough, both messages look the same when counted from 3rd byte in the <= 0.15 version and counted from the 2nd byte in 0.16.
Which after another look show that this is not true but can be explained by changed protobuf definitions. So the culprit is the prefix.
TM16 vs TM15:
PREFIX 0x26 vs 0x01,0x13

Nothing in the changelog hints on what might be going on here but it looks as if the previous varint approach was dropped in tm 0.16.
We have no idea where this change is coming from and a quick "help" in riot pointed to go-wire, honestely, we have no idea on what libraries to what in the tendermint ecosystem when it comes to the socket protocol.

Can you elaborate on that?

Rethink go-wire API

As we are working to design the cosmos-sdk API, I am writing up sample apps, and realized how confusing it can be to explain go-wire to newcomers. Not just funny things like needing to embed an interface in a struct, but also the api itself. I want to propose that we redesign the public API, hide most of the methods (which can be private), and adapt the rest to match a cleaner interface. I am not proposing here any fundamental changes to the feature set, except cleaning up how things work.

For example, look at the godoc: https://godoc.org/github.com/tendermint/go-wire

Compared with another binary encoding library: https://godoc.org/encoding/gob#pkg-index
Or even the extremely feature (and error) rich json encoding library: https://godoc.org/encoding/json#pkg-index

Not just the giant intro in the beginning, but that gob has about 90% fewer public methods and a very similar functionality (including registering implementations of interfaces). I am not suggesting to use gob instead (it is much larger than go-wire), but to model our API against something standard.

As a strawman, I would suggest something like this:

// registration multiple implementations for an interface, similar to go-data, but global
// multiple calls to Interface with the same base will return the same Registrar
type Registrar
  func Interface(base interface{}) Registrar
  func (r Registrar) RegisterImplementation(data interface{}, b byte) Registrar

// if we want to read/write to bytes - eg. ReadBinaryBytes and BinaryBytes
func Marshal(v interface{}) ([]byte, error)
func Unmarshal(data []byte, v interface{}) error

// if we want to read/write to streams of data - eg. ReadBinary and WriteBinary
type Decoder
  func NewDecoder(r io.Reader) *Decoder
  func (dec *Decoder) Decode(v interface{}) error
type Encoder
  func NewEncoder(w io.Writer) *Encoder
  func (enc *Encoder) Encode(v interface{}) error

Should RegisterInterface PanicSanity if re-registering a type?

Currently go-wire will let you overwrite the global serialisation info for a particularly interface type:

https://github.com/tendermint/go-wire/blob/master/reflect.go#L144

This opens up the possibility for some nasty import tree ordering bugs. Reorganising your code could lead to a different registration winning, leading to odd behaviour.

It seems like it should always be an error to try and re-register an interface, unless you explicitly specify you know what you are doing and really want to dynamically change the registered types. Personally I'd just make it a panic if we ever try to re-define but you could have OverrideRegisteredInterface if necessary.

Unify GetFoo and ReadFoo along with PutFoo and WriteFoo

the following functions have both a Read (from a Reader) and a Get (from a []byte buffer) variant

• GetByteSlice / ReadByteSlice
• GetBool / ReadBool
• GetInt16 / ReadInt16
• GetUint16 / ReadUint16
• GetInt23 / ReadInt32
• GetUint32 / ReadUint32
• GetInt64 / ReadInt64
• GetUint64 / ReadUint64
• GetVarint / ReadVarint
• GetUvarint / ReadUvarint
• GetString / ReadString

There's similar pairs of PutFoo and WriteFoo functions as well.

Each of these function pairs contains similar error checking code i.e. the following two snippets are nearly identical:

https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/byteslice.go#L54-L59

https://github.com/tendermint/go-wire/blob/5f88da3dbc1a72844e6dfaf274ce87f851d488eb/byteslice.go#L19-L26

This is a problem as it required the maintenance of the same sanity checks in two places and this code may diverge at some future time when one of the pairs is modified but not the other. There's no security issue but it's a code smell that can cause problems in the future.

Is it possible to merge the pairs above (i.e. by wrapping buffers in a bytes.NewReader)?

Side note: I don't see the Get variants being used very much, maybe they're dead code?