method handler crashed error for prestateTracer about celo-blockchain

[mikhail-hiryliuk]

@palango thanks for fixing the issue with error response!
considering that you don't support diffMode parameter yet, is there another way to trace native token balance changes? e.g on transaction example I provided. Is there a way to track how Celo token balances will be changed for two addresses?
since raw structLogs traces from default tracer don't provide such information

[palango]

@quick-pawiromitchel Thanks for reporting this. I could reproduce it.

The stacktrace is:

{"msg":"RPC method debug_traceCall crashed: runtime error: invalid memory address or nil pointer dereference
goroutine 25172668 [running]:
github.com/celo-org/celo-blockchain/rpc.(*callback).call.func1()
	github.com/celo-org/celo-blockchain/rpc/service.go:200 +0x89
panic({0x166b720, 0x244b1f0})
	runtime/panic.go:1038 +0x215
github.com/celo-org/celo-blockchain/eth/tracers.(*dbWrapper).pushObject.func1(0xc0703789c0)
	github.com/celo-org/celo-blockchain/eth/tracers/tracer.go:200 +0x9e
gopkg.in/olebedev/go-duktape%2ev3.goFunctionCall(0x46885a0)
	gopkg.in/olebedev/[email protected]/duktape.go:157 +0xb2
gopkg.in/olebedev/go-duktape%2ev3._Cfunc_duk_pcall_prop(0x46885a0, 0x0, 0x2)
	_cgo_gotypes.go:2330 +0x4c
gopkg.in/olebedev/go-duktape%2ev3.(*Context).PcallProp.func1(0x1, 0x0, 0x2)
	gopkg.in/olebedev/[email protected]/api.go:786 +0x65
gopkg.in/olebedev/go-duktape%2ev3.(*Context).PcallProp(0x30, 0x182f79b, 0x182e528)
	gopkg.in/olebedev/[email protected]/api.go:786 +0x19
github.com/celo-org/celo-blockchain/eth/tracers.(*Tracer).call(0xc06ce70c30, 0x1, {0x182f79b, 0x0}, {0xc071352668, 0x2, 0x10})
	github.com/celo-org/celo-blockchain/eth/tracers/tracer.go:643 +0xba
github.com/celo-org/celo-blockchain/eth/tracers.(*Tracer).CaptureState(0xc06ce70c30, 0x165bcc0, 0x0, 0x45, 0x1653020, 0xc090120930, 0x20, {0x0, 0x0, 0x0}, ...)
	github.com/celo-org/celo-blockchain/eth/tracers/tracer.go:732 +0x246
github.com/celo-org/celo-blockchain/core/vm.(*EVMInterpreter).Run(0xc070447b00, 0xc07040f2c0, {0xc07040a600, 0x24, 0x40}, 0x1)
	github.com/celo-org/celo-blockchain/core/vm/interpreter.go:257 +0xaeb
github.com/celo-org/celo-blockchain/core/vm.(*EVM).StaticCall(0xc070454a80, {0x1a46a00, 0xc070452930}, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}, ...)
	github.com/celo-org/celo-blockchain/core/vm/evm.go:425 +0x685
github.com/celo-org/celo-blockchain/core/vm/vmcontext.(*SharedEVMRunner).Query(0x0, {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}, ...)
	github.com/celo-org/celo-blockchain/core/vm/vmcontext/runner.go:102 +0xac
github.com/celo-org/celo-blockchain/contracts.(*BoundMethod).run(0xc00033fa10, {0x1a6fe58, 0xc01dde8698}, {0x17b9d60, 0xc070452900}, 0x1, 0x20301c, {0xc06bd5b820, 0x1, 0x1})
	github.com/celo-org/celo-blockchain/contracts/contract_call.go:103 +0x336
github.com/celo-org/celo-blockchain/contracts.(*BoundMethod).Query(...)
	github.com/celo-org/celo-blockchain/contracts/contract_call.go:76
github.com/celo-org/celo-blockchain/contracts.GetRegisteredAddress({0x1a6fe58, 0xc01dde8698}, {0xaa, 0xb7, 0x4a, 0x77, 0xd8, 0xce, 0x26, 0xd2, ...})
	github.com/celo-org/celo-blockchain/contracts/registry.go:23 +0x1b7
github.com/celo-org/celo-blockchain/contracts/reserve.TobinTax({0x1a6fe58, 0xc01dde8698}, {0x61, 0x64, 0xa, 0x8d, 0x48, 0xbc, 0xa2, 0x5, ...})
	github.com/celo-org/celo-blockchain/contracts/reserve/tobin_tax.go:29 +0x6a
github.com/celo-org/celo-blockchain/contracts/reserve.ComputeTobinTax({0x1a6fe58, 0xc01dde8698}, {0x61, 0x64, 0xa, 0x8d, 0x48, 0xbc, 0xa2, 0x5, ...}, ...)
	github.com/celo-org/celo-blockchain/contracts/reserve/tobin_tax.go:56 +0x71
github.com/celo-org/celo-blockchain/core/vm/vmcontext.TobinTransfer(0xc070454a80, {0x61, 0x64, 0xa, 0x8d, 0x48, 0xbc, 0xa2, 0x5, 0xba, ...}, ...)
	github.com/celo-org/celo-blockchain/core/vm/vmcontext/context.go:153 +0x145
github.com/celo-org/celo-blockchain/core/vm.(*EVM).Call(0xc070454a80, {0x1a46a00, 0xc070452888}, {0xe7, 0x5b, 0xb, 0xc5, 0xdd, 0xa1, 0xcb, ...}, ...)
	github.com/celo-org/celo-blockchain/core/vm/evm.go:230 +0x55d
github.com/celo-org/celo-blockchain/core.(*StateTransition).TransitionDb(0xc072cff2c0)
	github.com/celo-org/celo-blockchain/core/state_transition.go:568 +0xa99
github.com/celo-org/celo-blockchain/core.ApplyMessage(0xc072cff220, {0x1a8cbc0, 0xc072cff220}, 0xc094881560, {0x1a6fe98, 0xc070378980}, 0xef6c05985b384f23)
	github.com/celo-org/celo-blockchain/core/state_transition.go:242 +0x47f
github.com/celo-org/celo-blockchain/eth/tracers.(*API).traceTx(0xc006e20bb0, {0x1a695f8, 0xc080769280}, {0x1a8cbc0, 0xc072cff220}, 0xc0713550c8, {0x18d0c58, 0x18d0c78, 0xc073472d20, 0xc094881560, ...}, ...)
	github.com/celo-org/celo-blockchain/eth/tracers/api.go:938 +0x696
github.com/celo-org/celo-blockchain/eth/tracers.(*API).TraceCall(0xc006e20bb0, {0x1a695f8, 0xc080769280}, {0xc06ddb2ca8, 0xc06ddb2cc0, 0x0, 0x0, 0x0, 0x0, 0x0, ...}, ...)
	github.com/celo-org/celo-blockchain/eth/tracers/api.go:890 +0x7b0
reflect.Value.call({0xc000121180, 0xc00c865b18, 0x7f5640d2fa68}, {0x182ef37, 0x4}, {0xc0496e2780, 0x5, 0xc06f382500})
	reflect/value.go:556 +0x845
reflect.Value.Call({0xc000121180, 0xc00c865b18, 0x5ac0a8}, {0xc0496e2780, 0x5, 0x5})
	reflect/value.go:339 +0xc5
github.com/celo-org/celo-blockchain/rpc.(*callback).call(0xc00cac6840, {0x1a695f8, 0xc080769280}, {0xc0690b7370, 0xf}, {0xc06da03cc0, 0x3, 0x4db737})
	github.com/celo-org/celo-blockchain/rpc/service.go:206 +0x3f0
github.com/celo-org/celo-blockchain/rpc.(*handler).runMethod(0xc0662544d0, {0x1a695f8, 0xc080769280}, 0xc06fa31d50, 0x732da28dbcf68feb, {0xc06da03cc0, 0x0, 0x203024})
	github.com/celo-org/celo-blockchain/rpc/handler.go:389 +0x45
github.com/celo-org/celo-blockchain/rpc.(*handler).handleCall(0xc06d53db00, 0xc090120390, 0xc06fa31d50)
	github.com/celo-org/celo-blockchain/rpc/handler.go:337 +0x239
github.com/celo-org/celo-blockchain/rpc.(*handler).handleCallMsg(0xc06d53db00, 0xc000070c00, 0xc06fa31d50)
	github.com/celo-org/celo-blockchain/rpc/handler.go:298 +0xd1
github.com/celo-org/celo-blockchain/rpc.(*handler).handleMsg.func1(0xc090120390)
	github.com/celo-org/celo-blockchain/rpc/handler.go:139 +0x35
github.com/celo-org/celo-blockchain/rpc.(*handler).startCallProc.func1()
	github.com/celo-org/celo-blockchain/rpc/handler.go:226 +0xc5
created by github.com/celo-org/celo-blockchain/rpc.(*handler).startCallProc
	github.com/celo-org/celo-blockchain/rpc/handler.go:222 +0x92
","severity":"error","timestamp":"2023-04-19T13:44:52.564689564Z"}
{"conn":"127.0.0.1:48464","err":"method handler crashed","msg":"Served debug_traceCall","reqid":"0","severity":"warning","t":"14.83337ms","timestamp":"2023-04-19T13:44:52.564869265Z"}

[palango]

@quick-pawiromitchel Just to clarify, besides the crash which shouldn't happen, what outcome did you expect?

The example you provided sends a normal transaction without code, so the tracer's CaptureState method would never have been called because of this earlier check:
https://github.com/ethereum/go-ethereum/blob/eae3b1946a276ac099e0018fc792d9e8c3bfda6d/core/vm/interpreter.go#L136-L139

So now I wonder if this is an existing bug in the prestate tracer (that may have been fixed upstream in the meantime).

[mikhail-hiryliuk]

@quick-pawiromitchel thanks for submitting the issue!
@palango thanks for looking into that
Event though this is simple native token transaction without smart contract execution I'm expecting that rpc call with tracer: prestateTracer returns native token balance change for each address involved like geth client does.
I tried similar request on geth client and it works fine e.g

   {
    "method": "debug_traceCall",
     "params": [
              {
                "from": "0x61640A8D48Bca205BA91b16B0B7745e7aBc61084",
                "to": "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648",
                "value": "0x38D7EA4C68000"
              },
              "latest",
              {
              "tracer": "prestateTracer",
               "tracerConfig": {
                  "diffMode": true
                }
              }
            ],
            "id": 1,
            "jsonrpc": "2.0"
   }

got response

{
    "jsonrpc": "2.0",
    "id": 1,
    "result":
    {
        "post":
        {
            "0x61640a8d48bca205ba91b16b0b7745e7abc61084":
            {
                "balance": "0x1d138789aba2d0",
                "nonce": 20
            },
            "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648":
            {
                "balance": "0x38d7ea4c68000"
            }
        },
        "pre":
        {
            "0x61640a8d48bca205ba91b16b0b7745e7abc61084":
            {
                "balance": "0x20a1062e7222d0",
                "nonce": 19
            },
            "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648":
            {
                "balance": "0x0"
            }
        }
    }
}

Please let me know if something else is needed from my side to troubleshoot that.

[palango]

@quick-pawiromitchel @mikhail-hiryliuk
I suspect this is fixed by #2075. I see a different error message on mainnet without the fix, but get a trace with the fix. Do you have the possibility to test that branch?

Note that the diff mode has been added more recently, so we don't include it yet.

[palango]

I tested this again and #2075 fixes this problem.

The difference in error messages stems from the different versions.
On 1.7.3 (and 1.7.4) this error is shown:

$ http 127.0.0.1:8545 jsonrpc=2.0 id=0 method=debug_traceCall params:='[{"from": "0x61640A8D48Bca205BA91b16B0B7745e7aBc61084","to": "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648","value": "0x5af3107a4000"},"latest",{"tracer": "prestateTracer"}]'
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 101
Content-Type: application/json
Date: Thu, 27 Apr 2023 08:57:11 GMT

{
    "error": {
        "code": -32000,
        "message": "method handler crashed"
    },
    "id": "0",
    "jsonrpc": "2.0"
}

master already includes #2061, which fixes the crash.

$ http 127.0.0.1:8545 jsonrpc=2.0 id=0 method=debug_traceCall params:='[{"from": "0x61640A8D48Bca205BA91b16B0B7745e7aBc61084","to": "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648","value": "0x5af3107a4000"},"latest",{"tracer": "prestateTracer"}]'
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 177
Content-Type: application/json
Date: Thu, 27 Apr 2023 08:58:09 GMT

{
    "error": {
        "code": -32000,
        "message": "TypeError: cannot read property '0x61640a8d48bca205ba91b16b0b7745...' of null    in server-side tracer function 'result'"
    },
    "id": "0",
    "jsonrpc": "2.0"
}

And with #2075 we get the trace:

$  http 127.0.0.1:8545 jsonrpc=2.0 id=0 method=debug_traceCall params:='[{"from": "0x61640A8D48Bca205BA91b16B0B7745e7aBc61084","to": "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648","value": "0x5af3107a4000"},"latest",{"tracer": "prestateTracer"}]'
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 182
Content-Type: application/json
Date: Thu, 27 Apr 2023 08:59:06 GMT

{
    "id": "0",
    "jsonrpc": "2.0",
    "result": {
        "0x61640a8d48bca205ba91b16b0b7745e7abc61084": {
            "balance": "0x7cb35c62f0ad5619",
            "code": "0x",
            "nonce": 16,
            "storage": {}
        },
        "0xe75b0bc5dda1cbbac7075de5b1a39e9755938648": {
            "balance": "0x0",
            "code": "0x",
            "nonce": 0,
            "storage": {}
        }
    }
}

[palango]

@palango thanks for fixing the issue with error response! considering that you don't support diffMode parameter yet, is there another way to trace native token balance changes? e.g on transaction example I provided. Is there a way to track how Celo token balances will be changed for two addresses? since raw structLogs traces from default tracer don't provide such information

@mikhail-hiryliuk Not that I'm aware unfortunately. However, once the PR that merges native (call)tracing is merged is should be relatively simple to merge the native prestate tracer with the diff mode feature.