GithubHelp home page GithubHelp logo

heidihoward / ios Goto Github PK

View Code? Open in Web Editor NEW
47.0 8.0 7.0 45.87 MB

Reliable distributed agreement service for the cloud

License: MIT License

Go 99.28% Shell 0.72%
consensus fault-tolerance consistency flexible-paxos reliability agreement

ios's Issues

runtime: out of memory due to JSON marshal/unmarshal

There are various memory issues with JSON marshal/unmarshal. At the moment, we do this either in a strange verbose manner or has been replicated by a placeholder to prevent errors like this:

~/go/bin/server -id=0
fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x5ddf0c, 0x16)
	/usr/local/go/src/runtime/panic.go:566 +0x95
runtime.sysMap(0xc440500000, 0x20000000, 0xa00001f700, 0x6d64b8)
	/usr/local/go/src/runtime/mem_linux.go:219 +0x1d0
runtime.(*mheap).sysAlloc(0x6bda40, 0x20000000, 0x34e3f)
	/usr/local/go/src/runtime/malloc.go:407 +0x37a
runtime.(*mheap).grow(0x6bda40, 0x10000, 0x0)
	/usr/local/go/src/runtime/mheap.go:726 +0x62
runtime.(*mheap).allocSpanLocked(0x6bda40, 0x10000, 0x0)
	/usr/local/go/src/runtime/mheap.go:630 +0x4f2
runtime.(*mheap).allocStack(0x6bda40, 0x10000, 0x40f0fc)
	/usr/local/go/src/runtime/mheap.go:597 +0x62
runtime.stackalloc(0x20000000, 0xc41fd57b00, 0x100, 0x40, 0x2000, 0x7f0d9b218710)
	/usr/local/go/src/runtime/stack.go:395 +0x2ed
runtime.copystack(0xc4200001a0, 0x20000000, 0x1)
	/usr/local/go/src/runtime/stack.go:839 +0x83
runtime.newstack()
	/usr/local/go/src/runtime/stack.go:1070 +0x370
runtime.morestack()
	/usr/local/go/src/runtime/asm_amd64.s:366 +0x7f

goroutine 1 [copystack]:
encoding/json.(*decodeState).object(0xc42550ee10, 0x5aa960, 0xc4254fd668, 0x16)
	/usr/local/go/src/encoding/json/decode.go:557 fp=0xc430500488 sp=0xc430500480
encoding/json.(*decodeState).value(0xc42550ee10, 0x5aa960, 0xc4254fd668, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430500508 sp=0xc430500488
encoding/json.(*decodeState).unmarshal(0xc42550ee10, 0x5aa960, 0xc4254fd668, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430500580 sp=0xc430500508
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd668, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc4305005c8 sp=0xc430500580
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd650, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430500610 sp=0xc4305005c8
encoding/json.(*decodeState).object(0xc42550ed20, 0x5aa960, 0xc4254fd650, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc4305008a0 sp=0xc430500610
encoding/json.(*decodeState).value(0xc42550ed20, 0x5aa960, 0xc4254fd650, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430500920 sp=0xc4305008a0
encoding/json.(*decodeState).unmarshal(0xc42550ed20, 0x5aa960, 0xc4254fd650, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430500998 sp=0xc430500920
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd650, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc4305009e0 sp=0xc430500998
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd638, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430500a28 sp=0xc4305009e0
encoding/json.(*decodeState).object(0xc42550ec30, 0x5aa960, 0xc4254fd638, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430500cb8 sp=0xc430500a28
encoding/json.(*decodeState).value(0xc42550ec30, 0x5aa960, 0xc4254fd638, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430500d38 sp=0xc430500cb8
encoding/json.(*decodeState).unmarshal(0xc42550ec30, 0x5aa960, 0xc4254fd638, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430500db0 sp=0xc430500d38
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd638, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430500df8 sp=0xc430500db0
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd620, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430500e40 sp=0xc430500df8
encoding/json.(*decodeState).object(0xc42550eb40, 0x5aa960, 0xc4254fd620, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc4305010d0 sp=0xc430500e40
encoding/json.(*decodeState).value(0xc42550eb40, 0x5aa960, 0xc4254fd620, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430501150 sp=0xc4305010d0
encoding/json.(*decodeState).unmarshal(0xc42550eb40, 0x5aa960, 0xc4254fd620, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc4305011c8 sp=0xc430501150
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd620, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430501210 sp=0xc4305011c8
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd608, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430501258 sp=0xc430501210
encoding/json.(*decodeState).object(0xc42550ea50, 0x5aa960, 0xc4254fd608, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc4305014e8 sp=0xc430501258
encoding/json.(*decodeState).value(0xc42550ea50, 0x5aa960, 0xc4254fd608, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430501568 sp=0xc4305014e8
encoding/json.(*decodeState).unmarshal(0xc42550ea50, 0x5aa960, 0xc4254fd608, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc4305015e0 sp=0xc430501568
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd608, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430501628 sp=0xc4305015e0
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd5f0, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430501670 sp=0xc430501628
encoding/json.(*decodeState).object(0xc42550e960, 0x5aa960, 0xc4254fd5f0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430501900 sp=0xc430501670
encoding/json.(*decodeState).value(0xc42550e960, 0x5aa960, 0xc4254fd5f0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430501980 sp=0xc430501900
encoding/json.(*decodeState).unmarshal(0xc42550e960, 0x5aa960, 0xc4254fd5f0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc4305019f8 sp=0xc430501980
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd5f0, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430501a40 sp=0xc4305019f8
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd5d8, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430501a88 sp=0xc430501a40
encoding/json.(*decodeState).object(0xc42550e870, 0x5aa960, 0xc4254fd5d8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430501d18 sp=0xc430501a88
encoding/json.(*decodeState).value(0xc42550e870, 0x5aa960, 0xc4254fd5d8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430501d98 sp=0xc430501d18
encoding/json.(*decodeState).unmarshal(0xc42550e870, 0x5aa960, 0xc4254fd5d8, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430501e10 sp=0xc430501d98
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd5d8, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430501e58 sp=0xc430501e10
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd5c0, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430501ea0 sp=0xc430501e58
encoding/json.(*decodeState).object(0xc42550e780, 0x5aa960, 0xc4254fd5c0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430502130 sp=0xc430501ea0
encoding/json.(*decodeState).value(0xc42550e780, 0x5aa960, 0xc4254fd5c0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc4305021b0 sp=0xc430502130
encoding/json.(*decodeState).unmarshal(0xc42550e780, 0x5aa960, 0xc4254fd5c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430502228 sp=0xc4305021b0
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd5c0, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430502270 sp=0xc430502228
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd5a8, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc4305022b8 sp=0xc430502270
encoding/json.(*decodeState).object(0xc42550e690, 0x5aa960, 0xc4254fd5a8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430502548 sp=0xc4305022b8
encoding/json.(*decodeState).value(0xc42550e690, 0x5aa960, 0xc4254fd5a8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc4305025c8 sp=0xc430502548
encoding/json.(*decodeState).unmarshal(0xc42550e690, 0x5aa960, 0xc4254fd5a8, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430502640 sp=0xc4305025c8
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd5a8, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430502688 sp=0xc430502640
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd590, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc4305026d0 sp=0xc430502688
encoding/json.(*decodeState).object(0xc42550e5a0, 0x5aa960, 0xc4254fd590, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430502960 sp=0xc4305026d0
encoding/json.(*decodeState).value(0xc42550e5a0, 0x5aa960, 0xc4254fd590, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc4305029e0 sp=0xc430502960
encoding/json.(*decodeState).unmarshal(0xc42550e5a0, 0x5aa960, 0xc4254fd590, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430502a58 sp=0xc4305029e0
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd590, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430502aa0 sp=0xc430502a58
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd578, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430502ae8 sp=0xc430502aa0
encoding/json.(*decodeState).object(0xc42550e4b0, 0x5aa960, 0xc4254fd578, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430502d78 sp=0xc430502ae8
encoding/json.(*decodeState).value(0xc42550e4b0, 0x5aa960, 0xc4254fd578, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430502df8 sp=0xc430502d78
encoding/json.(*decodeState).unmarshal(0xc42550e4b0, 0x5aa960, 0xc4254fd578, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430502e70 sp=0xc430502df8
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd578, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430502eb8 sp=0xc430502e70
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd560, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430502f00 sp=0xc430502eb8
encoding/json.(*decodeState).object(0xc42550e3c0, 0x5aa960, 0xc4254fd560, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430503190 sp=0xc430502f00
encoding/json.(*decodeState).value(0xc42550e3c0, 0x5aa960, 0xc4254fd560, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430503210 sp=0xc430503190
encoding/json.(*decodeState).unmarshal(0xc42550e3c0, 0x5aa960, 0xc4254fd560, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430503288 sp=0xc430503210
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd560, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc4305032d0 sp=0xc430503288
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd548, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430503318 sp=0xc4305032d0
encoding/json.(*decodeState).object(0xc42550e2d0, 0x5aa960, 0xc4254fd548, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc4305035a8 sp=0xc430503318
encoding/json.(*decodeState).value(0xc42550e2d0, 0x5aa960, 0xc4254fd548, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430503628 sp=0xc4305035a8
encoding/json.(*decodeState).unmarshal(0xc42550e2d0, 0x5aa960, 0xc4254fd548, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc4305036a0 sp=0xc430503628
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd548, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc4305036e8 sp=0xc4305036a0
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd530, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430503730 sp=0xc4305036e8
encoding/json.(*decodeState).object(0xc42550e1e0, 0x5aa960, 0xc4254fd530, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc4305039c0 sp=0xc430503730
encoding/json.(*decodeState).value(0xc42550e1e0, 0x5aa960, 0xc4254fd530, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430503a40 sp=0xc4305039c0
encoding/json.(*decodeState).unmarshal(0xc42550e1e0, 0x5aa960, 0xc4254fd530, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430503ab8 sp=0xc430503a40
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd530, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430503b00 sp=0xc430503ab8
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd518, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430503b48 sp=0xc430503b00
encoding/json.(*decodeState).object(0xc42550e0f0, 0x5aa960, 0xc4254fd518, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430503dd8 sp=0xc430503b48
encoding/json.(*decodeState).value(0xc42550e0f0, 0x5aa960, 0xc4254fd518, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430503e58 sp=0xc430503dd8
encoding/json.(*decodeState).unmarshal(0xc42550e0f0, 0x5aa960, 0xc4254fd518, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430503ed0 sp=0xc430503e58
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd518, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430503f18 sp=0xc430503ed0
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd500, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430503f60 sp=0xc430503f18
encoding/json.(*decodeState).object(0xc42550e000, 0x5aa960, 0xc4254fd500, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc4305041f0 sp=0xc430503f60
encoding/json.(*decodeState).value(0xc42550e000, 0x5aa960, 0xc4254fd500, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430504270 sp=0xc4305041f0
encoding/json.(*decodeState).unmarshal(0xc42550e000, 0x5aa960, 0xc4254fd500, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc4305042e8 sp=0xc430504270
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd500, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430504330 sp=0xc4305042e8
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd4e8, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430504378 sp=0xc430504330
encoding/json.(*decodeState).object(0xc42550def0, 0x5aa960, 0xc4254fd4e8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430504608 sp=0xc430504378
encoding/json.(*decodeState).value(0xc42550def0, 0x5aa960, 0xc4254fd4e8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430504688 sp=0xc430504608
encoding/json.(*decodeState).unmarshal(0xc42550def0, 0x5aa960, 0xc4254fd4e8, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430504700 sp=0xc430504688
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd4e8, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430504748 sp=0xc430504700
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd4d0, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430504790 sp=0xc430504748
encoding/json.(*decodeState).object(0xc42550de00, 0x5aa960, 0xc4254fd4d0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430504a20 sp=0xc430504790
encoding/json.(*decodeState).value(0xc42550de00, 0x5aa960, 0xc4254fd4d0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430504aa0 sp=0xc430504a20
encoding/json.(*decodeState).unmarshal(0xc42550de00, 0x5aa960, 0xc4254fd4d0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430504b18 sp=0xc430504aa0
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd4d0, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430504b60 sp=0xc430504b18
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd4b8, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430504ba8 sp=0xc430504b60
encoding/json.(*decodeState).object(0xc42550dd10, 0x5aa960, 0xc4254fd4b8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430504e38 sp=0xc430504ba8
encoding/json.(*decodeState).value(0xc42550dd10, 0x5aa960, 0xc4254fd4b8, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc430504eb8 sp=0xc430504e38
encoding/json.(*decodeState).unmarshal(0xc42550dd10, 0x5aa960, 0xc4254fd4b8, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430504f30 sp=0xc430504eb8
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd4b8, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430504f78 sp=0xc430504f30
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd4a0, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc430504fc0 sp=0xc430504f78
encoding/json.(*decodeState).object(0xc42550dc20, 0x5aa960, 0xc4254fd4a0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:562 +0x1cb5 fp=0xc430505250 sp=0xc430504fc0
encoding/json.(*decodeState).value(0xc42550dc20, 0x5aa960, 0xc4254fd4a0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:368 +0x2cc fp=0xc4305052d0 sp=0xc430505250
encoding/json.(*decodeState).unmarshal(0xc42550dc20, 0x5aa960, 0xc4254fd4a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:169 +0x1fa fp=0xc430505348 sp=0xc4305052d0
encoding/json.Unmarshal(0xc42008029c, 0x2, 0x4, 0x5aa960, 0xc4254fd4a0, 0x4, 0xc42008029c)
	/usr/local/go/src/encoding/json/decode.go:97 +0x102 fp=0xc430505390 sp=0xc430505348
github.com/heidi-ann/ios/services.(*dummy).UnmarshalJSON(0xc4254fd488, 0xc42008029c, 0x2, 0x4, 0x0, 0x7f0d9dce8a78)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/services/dummy.go:26 +0x6c fp=0xc4305053d8 sp=0xc430505390
...additional frames elided...

Segmentation violation in test suite

Test suite is currently failing due to the following:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x4eb15b]

goroutine 19 [running]:
panic(0x5a6120, 0xc4200100a0)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/heidi-ann/ios/app.(*StateMachine).Apply(0xc4200143c0, 0x0, 0x1, 0x0, 0xc4200148a0, 0x11, 0x0, 0x0, 0x0, 0x0, ...)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/app/app.go:32 +0x3bb
github.com/heidi-ann/ios/consensus.runParticipant(0xc420017940, 0xc4200d2000, 0x0, 0x1, 0x186a0, 0x0, 0x1, 0x0, 0x1, 0x3e8, ...)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/consensus/participant.go:69 +0x1379
github.com/heidi-ann/ios/consensus.Init(0xc4200d2000, 0x0, 0x1, 0x186a0, 0x0, 0x1, 0x0, 0x1, 0x3e8, 0x5daa3e, ...)
	/home/ubuntu/go/src/github.com/heidi-ann/ios/consensus/consensus.go:63 +0x575
created by main.main
	/home/ubuntu/go/src/github.com/heidi-ann/ios/server/server.go:77 +0xe4b

I will take a look into it now

Efficient representation of NO-OPS

Currently, NO-OPS are represented explicitly like all other log entries, this is not necessary and could be done much more efficiently

add a configuration file validator

Sometimes client, server or workload configuration files are wrong/incorrectly formatted. Currently, we need to start Ios and wait for some exception to discover this. Would be useful to add a configuration verifier which hooks into functions like config.ParseServerConfig to check config files.

Add cluster ID to all I/O

Currently, multiple deployments of Ios running across the same hosts risk interfering with each other. This could be address by assigning each deployment at random ID and checking this ID when receiving messages/reading from disk

Add -fresh option to server

Add -fresh option to server command line to instruct the Ios server to ignore and overwrite persistent storage if present. This will save the need to run rm persistent_* before starting a fresh Ios instance.

Add support for Windows

Currently Ios only builds Linux due to use of syscalls such as Fallocate, Fdatasync and directI/O

Switch configuration file parsing library

Currently, we use gvfg for parsing configuration files however it does not support optional parameters. The result is that Ios configuration files have many parameters most of which are useless to a standard developer

concurrent map writes in window.go

Bug found on OSX running the 3 server deployment from example3.conf with test clients.

fatal error: concurrent map writes

goroutine 51 [running]:
runtime.throw(0x28a3b0, 0x15)
	/usr/local/go/src/runtime/panic.go:547 +0x90 fp=0xc820b175e0 sp=0xc820b175c8
runtime.mapassign1(0x1c60c0, 0xc82000a600, 0xc820b17748, 0xc820b1771e)
	/usr/local/go/src/runtime/hashmap.go:445 +0xb1 fp=0xc820b17688 sp=0xc820b175e0
github.com/heidi-ann/ios/consensus.runMaster(0x0, 0xffffffffffffffff, 0x1, 0xc8200fc000, 0x0, 0x3, 0x186a0, 0x0, 0x64, 0x0, ...)
	/Users/heidi/go/src/github.com/heidi-ann/ios/consensus/master.go:145 +0xc6b fp=0xc820b17b50 sp=0xc820b17688
github.com/heidi-ann/ios/consensus.monitorMaster(0xc820094300, 0xc8200fc000, 0x0, 0x3, 0x186a0, 0x0, 0x64, 0x0, 0xa, 0x64, ...)
	/Users/heidi/go/src/github.com/heidi-ann/ios/consensus/master.go:14 +0x1cd fp=0xc820b17f20 sp=0xc820b17b50
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc820b17f28 sp=0xc820b17f20
created by github.com/heidi-ann/ios/consensus.Init
	/Users/heidi/go/src/github.com/heidi-ann/ios/consensus/consensus.go:62 +0x6b9

goroutine 1 [chan receive, 1 minutes]:
main.main()
	/Users/heidi/go/src/github.com/heidi-ann/ios/server/server.go:90 +0x11f3

I will look into it further now

Fix builds on OSX

/unix/wal.go uses syscalls such as Fallocate, Fdatasync which are not available on OSX so Ios will not build. This could be fixed by adding an wal_osx.go which doesn't use these calls and building the correct version depending on the host

Quick negative response for malformed requests

Currently, if a request for an application state machine is malformed then it must be replicated by consensus system before being applied to the state machine and detected. Server instead could check formatting before replication

Message dispatch issue on master failure

There exists a serious bug in the IO abstraction used in server.go.

To reproduce, set scripts/failure_testing.sh to kill node 0, run it and observe that the New View RPC is not sent by node 1 to node 2.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.