testing: Figure out how to scrub sensitive information from RPCs about go-cloud

[zombiezen]

(This is broadly applicable, not just to runtimeconfig.)

[cflewis]

Looks like messages are matched exactly.

The simplest thing is probably to inspect the protos and scrub them when they go in and when they go out.

This might be what we should do for HTTP too; just have a wrapper transport that scrubs rather than reloading the file.

[cflewis]

It is not clear to me that rpcreplay can be chained as it doesn't expose the invoking functions that it uses for interception. It's possible that DialOptions handle chained calls to WithUnaryInterceptor correctly, but I am skeptical. I'll try and see.

[cflewis]

As feared, chaining WithUnaryInterceptor is not possible. I will need to open a PR on rpcreplay that exposes the interceptors.

[cflewis]

Exposing the interceptors doesn't help. The invoker is what sends the RPC upstream. The invoker is passed by GRPC, so there's no way to wrap it. Without doing major surgery on rpcreplay itself (which we could do by just forking it), the only thing to do is to expose the entry struct so entries in the replay file can be unmarshaled, load the replay file, scrub the protos and then save them into a new file, delete the old one then move the new file into place.

I do not like either option.

[cflewis]

This remains more involved than expected, but progress is being made.

@jba recommended modifying rpcreplay to take callbacks for before/after messages are sent. That I have done, but am not going to send the change until I'm confident what I did was correct.

It looks like NewGCPDialOptions will need to take a scrubber as an argument that is passed by the tests. The reason the tests need to define the scrubber is that you don't know what the possible protos are beforehand, nor where possible sensitive data might be.