processout / grpc-go-pool

grpc connection pool

ProcessOut Go

Installation

go get gopkg.in/processout.v4

Versioning

Versioning is done using gopkg.in, available at gopkg.in/processout.v4.

grpc-go-pool's Issues

Why do you lock Pool.clients?

Hi, I'm a Golang beginner and I learn that chan in Golang is goroutine safe. So may I ask why you lock the clients in (p *Pool) getClients()?

func (p *Pool) getClients() chan ClientConn {
    p.mu.RLock()
    defer p.mu.RUnlock()
    return p.clients
}

I don't find a function to put conn to pool when business finished.

Ths. I don't find a function to put conn to pool when business finished. In this case, it doesn't need to do so or i should write it myself?

ClientConn needs to be protected by mutex

Race detector complains:

Write at 0x00c0000b63c0 by goroutine 104:
  github.com/processout/grpc-go-pool.(*ClientConn).Close()
      /home/mic/go/pkg/mod/github.com/processout/[email protected]/pool.go:236 +0x344
  github.com/processout/grpc-go-pool.(*ClientConn).Close-fm()
      /home/mic/go/pkg/mod/github.com/processout/[email protected]/pool.go:195 +0x39

Previous read at 0x00c0000b63c0 by goroutine 101:
  github.com/processout/grpc-go-pool.(*ClientConn).Close()
      /home/mic/go/pkg/mod/github.com/processout/[email protected]/pool.go:220 +0x204
  github.com/processout/grpc-go-pool.(*ClientConn).Close-fm()
      /home/mic/go/pkg/mod/github.com/processout/[email protected]/pool.go:195 +0x39

maxLifeDuration should check after time initiated

Not sure if I got it right, but for my understanding - setting maxLifeDuration should result in unhealthy connection only if its After(time.Now()). should this be fixed ?

grpc-go-pool/pool.go

Line 194 in 6f31a97

if maxDuration > 0 && c.timeInitiated.Add(maxDuration).Before(time.Now()) {

Zombie connections when calling Close() after marking connection as unhealthy

We discovered that by marking a connection unhealthy in the pool the grpc connection is not closed after calling the wrapper Close() method. This can lead to multiple "zombie" connections alive and since the grpc reuses alive connections this will make the subsequent requests to fail.

I'v made a pr with the fix, I just cant push my branch to open it

Context Canceled Error

I'm getting

rpc error: code = Canceled desc = context canceled

This happened in the line

// client.StoryConn is a *grpcpool.Pool type
conn, err := client.StoryConn.Get(ctx)
if err != nil {
	logger.Err("Error: ", err)
}
defer conn.Close()

clientConn := buff.NewStoryServiceClient(conn.ClientConn)
result, err := clientConn.GetStories(ctx, &buff.GetStoryRequest{Any: "any"}) << error here

What could be causing the cancellation of the context of a gRPC call except timeouts?

No multiplexing on client connections.

Wanted to highlight a problem with the current implementation involving channels.
The way connections are used:-

1. Connections are created and pushed to the channel
2. Whenever a request is to be served, a connection is obtained from the channel, request is served and then connection is pushed back to the channel.

The problem with this approach is that if there are n connections in channel, at max we can only serve n requests concurrently because of the nature of the implementation which defies the core property on which gRPC was built ==> multiplexing.

I encountered a similar problem in my org where we had to cater a throughput of 300K RPS, which seemed unattainable with current implementation along with some other necessary handlings.

We came up with a custom solution of our own which abstracts all of the problems we faced and with this solution we were able to serve 300K RPS with 0 overhead because of custom code.

I have documented my entire journey of the problems we faced and the solution we coded to cater the same. I am posting it here with a hope that it helps anyone stuck with similar problem.

Medium Article

maxLifeDuration prematurely closes the connection

If MaxLifeDuration is set, connection was only reused once and gets closed early as during cloning of ClienConn

grpc-go-pool/pool.go

Line 205 in 0835618

wrapper := ClientConn{

, timeInitiated was never copied, hence the following condition

if maxDuration > 0 && c.timeInitiated.Add(maxDuration).Before(time.Now()) {
		c.Unhealthy()
	}

gets true next time the Close().

Fix is to copy the timeInitiated from the original object to the clone.Can someone please review #12 for the fix? Thanks

Upgrade for Go Modules / Go 1.13

Any chance this can be upgraded for Go Modules?

The "RPC error: code = unavailable desc = transport is closing" error occurred in the connection pool

The "RPC error: code = unavailable desc = transport is closing" error occurred in the connection pool. Is it because the connection pool does not detect the status of grpc connections?

Cannot use *grpcpool.ClientConn as type *grpc.ClientConn

> go build
./main.go:31:24: cannot use conn (type *grpcpool.ClientConn) as type *grpc.ClientConn in argument to testproto.NewFooClient

Above incompatible types error occured when build. Here's the code:
go version go1.11.5 linux/amd64

package main

import (
	"context"
	"github.com/processout/grpc-go-pool"
	"google.golang.org/grpc"
	"log"
	pb "/local/path/proto"
	"time"
)

func newConnection() (*grpc.ClientConn, error) {
	conn, err := grpc.Dial("127.0.0.1:10000", grpc.WithInsecure())
	if err != nil {
		log.Fatal("Connect to service failed. ", err)
		return nil, err
	}
	log.Println("Connected to service")
	return conn, nil
}

var connPool *grpcpool.Pool

func InitPool() (err error) {
	connPool, err = grpcpool.New(newConnection, 5, 5, time.Second)
	return
}

func main() {
        InitPool()
	conn, _ := connPool.Get(context.Background())
	cli := pb.NewFooClient(conn)
	log.Println("done")
}

Does the pool support blocking get and other advanced features in Vitess pools?

Please release a new tag including the recent codes in master

Hi, we need the FactoryWithContext committed in the master branch, please release a new tag including the recent codes in master. Thanks a lot.

How to use it?

Is there some example?

Re-using connections when all are checked out?

Is it possible to re-use a connection (multiplexing) when all of the clients are checked out? The use case is that I want to do some minimal client side load balancing while not having to reopen a new connection every single time.

Should `Factory` (function type creating a grpc client) accept a context?

Should we allow the Factory (function type creating a grpc client) to accept a context so that calls to Factory can timeout or be cancelled?

Proposed changes to pool.go.

Before:

// Factory is a function type creating a grpc client
type Factory func() (*grpc.ClientConn, error)

...

// Get will return the next available client. If capacity
// has not been reached, it will create a new one using the factory. Otherwise,
// it will wait till the next client becomes available or a timeout.
// A timeout of 0 is an indefinite wait
func (p *Pool) Get(ctx context.Context) (*ClientConn, error) {
	...

	var err error
	if wrapper.ClientConn == nil {
		wrapper.ClientConn, err = p.factory()
		if err != nil {
			// If there was an error, we want to put back a placeholder
			// client in the channel
			clients <- ClientConn{
				pool: p,
			}
		}
		// This is a new connection, reset its initiated time
		wrapper.timeInitiated = time.Now()
	}

	return &wrapper, err
}

After:

// Factory is a function type creating a grpc client
type Factory func(context.Context) (*grpc.ClientConn, error)

...

// Get will return the next available client. If capacity
// has not been reached, it will create a new one using the factory. Otherwise,
// it will wait till the next client becomes available or a timeout.
// A timeout of 0 is an indefinite wait
func (p *Pool) Get(ctx context.Context) (*ClientConn, error) {
	...

	var err error
	if wrapper.ClientConn == nil {
		wrapper.ClientConn, err = p.factory(ctx)
		if err != nil {
			// If there was an error, we want to put back a placeholder
			// client in the channel
			clients <- ClientConn{
				pool: p,
			}
		}
		// This is a new connection, reset its initiated time
		wrapper.timeInitiated = time.Now()
	}

	return &wrapper, err
}