grpc-proxy about grpc-tools

[Arnold1]

@bradleyjkemp I will checkout the branch - run a test and let you know - sounds good?

[bradleyjkemp]

Hey @Arnold1, yes grpc-proxy can probably help you here.
grpc-proxy will handle the forwarding of the request/response and you can ad your own Interceptor function(which has full control to modify the request/response before it is forwarded) to rewrite the request headers however you want.

[Arnold1]

@bradleyjkemp thanks for the info.

I cannot use gRPC StreamClientInterceptor btw. its grpc request/response which is called Unary RPC. would that still work?

it looks like only streaming is supported? https://github.com/bradleyjkemp/grpc-tools/blob/master/grpc-proxy/grpc_handler.go#L174

what I really need is a grpc.UnaryHandler?
https://godoc.org/google.golang.org/grpc#UnaryHandler

[bradleyjkemp]

Unary interceptors are just a special case of Stream interceptors 🙂 (they're just a stream with only one message sent in each direction)

grpc-proxy absolutely handles unary request/response methods but you're right this probably isn't the easiest thing to implement at the moment. I'll look into adding a config option for you to add a UnaryInterceptor

[Arnold1]

@bradleyjkemp awesome. is it a lot of work?

[bradleyjkemp]

Very simple 🙂 I think #83 implements all you need: you can construct a grpc-proxy and use the option WithUnaryInterceptor(yourInterceptor)

Let me know if you think you would need anything else?

[Arnold1]

@bradleyjkemp seems to work, except I don't see logs from my intercept func (I expected to see: "request called" ... I only see "start" which is from func main) ... any idea?

$ ./grpc-proxy -port=8080 -destination=localhost:8081 -log_level=debug
start...
INFO[0000] Listening on 127.0.0.1:8080                  
INFO[0000] Not intercepting TLS connections             
DEBU[0054] Got connection from address 127.0.0.1:29524  
DEBU[0054] Handling gRPC request .... 
DEBU[0054] Dialing new connection to localhost:8081      =connpool

package main

import (
	"flag"
	"fmt"
	"github.com/bradleyjkemp/grpc-tools/grpc-proxy"
	"google.golang.org/grpc"
	"context"
)

func main() {
	fmt.Printf("start...\n")

	grpc_proxy.RegisterDefaultFlags()
	flag.Parse()
	proxy, _ := grpc_proxy.New(
        grpc_proxy.WithUnaryInterceptor(intercept),
        grpc_proxy.DefaultFlags(),
    )
    proxy.Start()
}

func intercept(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
	fmt.Println(info.FullMethod)

	fmt.Printf("request called....\n")
	....
}

it looks like the intercept is not called at all...ideas?

[Arnold1]

@bradleyjkemp any idea?

[bradleyjkemp]

Ah sorry @Arnold1 I'm not sure this will work: because of how gRPC works, the server only knows that an RPC is unary if it knows the RPC definition (i.e. it has the .proto file)

For grpc-proxy it knows nothing about any RPC so has to be as generic as possible. This means it doesn't know whether the RPC it's handling is Unary or Streaming so has to assume that it's streaming.

Are you able to write your handler as a StreamInterceptor? It's definitely not the nicest abstraction but you can take a look at some examples here for how to wrap a ServerStream and intercept the messages from it: https://github.com/grpc-ecosystem/go-grpc-middleware/blob/master/tags/interceptors.go#L45

I think for your use case you'd just need to override custom RecvMsg to read the original message, unmarshal it, modify the proto as required, re-marshal the message and return it.

I'll also have a think about how this can be improved. For example, it would be nice if you could register a gRPC service handler with grpc-proxy and it could handle this for you

[bradleyjkemp]

This is the relevant part of the gRPC server if you were interested: https://github.com/grpc/grpc-go/blob/fb2e5cdc8543ff38ef5624eec945856ffcc3610a/server.go#L1285

If the service is not known then it always uses a streaming RPC handler (and so only streaming interceptors will run)

[Arnold1]

@bradleyjkemp can I overwrite/change the grpc-proxy code so it assumes that its Unary by default? how to do that? will it need to load the proto file?

@bradleyjkemp using the StreamInterceptor which unmarshals it for each request will be costly I think? how can I read the metadata (headers) from the request in the streaming interceptor? I want to read the metadata and based on a header I want to forward the request to port x or port y. is that possible?

I tried:

func main() {
	grpc_proxy.RegisterDefaultFlags()
	flag.Parse()
	proxy, _ := grpc_proxy.New(
        grpc_proxy.WithInterceptor(intercept),
        grpc_proxy.DefaultFlags(),
    )
    proxy.Start()
}

// func WithInterceptor(interceptor grpc.StreamServerInterceptor) Configurator {
// type StreamServerInterceptor func(srv interface{}, ss ServerStream, info *StreamServerInfo, handler StreamHandler) error
// type StreamHandler func(srv interface{}, stream ServerStream) error
func intercept(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
	fmt.Println(info.FullMethod)
	return handler(srv, ss)
}

and see logs:

$ ./grpc-proxy -port=8502 -destination=localhost:8081 -log_level=debug
INFO[0000] Listening on 127.0.0.1:8080                  
INFO[0000] Not intercepting TLS connections             
DEBU[0036] Got connection from address 127.0.0.1:32708  
DEBU[0036] Handling gRPC request /tensorflow.serving.PredictionService/Predict 
/tensorflow.serving.PredictionService/Predict
DEBU[0036] Dialing new connection to localhost:8081      =connpool
DEBU[0036] Handling gRPC request /tensorflow.serving.PredictionService/Predict 
/tensorflow.serving.PredictionService/Predict

[bradleyjkemp]

Unmarshalling on each request won't be very expensive. If you're just needing to read the headers/metadata then you don't even need to unmarshal the message.

What is it you're trying to accomplish? If you're just trying to route requests based on the header then grpc-proxy probably isn't the best choice: Envoy or some other Layer 7 proxy might be better.

grpc-proxy would only be necessary if you want to a modify the request/response messages

[Arnold1]

@bradleyjkemp ok I will try envoy as well.

than I want to try the following with grpc-proxy:

• read the metadata header and modify a field in the request - for unary RPC.

[Arnold1]

@bradleyjkemp hi, any infos on that?

I have the following scenario:
grpc client -> proxy -> grpc server (Tensorflow serving)

i get a request from the grpc client and need to call 2 requests to the grpc server and check the 2 responses and return one of them back to the grpc client

for grpc unary

[bradleyjkemp]

I think for your use case grpc-proxy isn't the best fit (or at least would take quite a bit of work to add the features you want).

As you already know exactly what gRPC service interface you need to implement it'd probably be easiest to write your own server for this. Assuming you can configure your client, there's no need for grpc-proxy's interception features

[Arnold1]

@bradleyjkemp how do you suggest to implement it? what do you mean by "Assuming you can configure your client, there's no need for grpc-proxy's interception features" ?

[bradleyjkemp]

Assuming you want to implement the Tensorflow MasterService (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/data/service/master.proto#L84), you just write a gRPC server that implements those methods.

When the methods are called you do whatever modifications you want and then forward on the request to whatever the real server is.

As long as your client has a setting to choose the server this will work fine (i.e. you can point it at your proxy instead of the real server)