jgaskins / grpc Goto Github PK

Official GRPC clients stop sending data after a few thousand frames.

Seems that something about the update_window_for method isn't actually updating the flow-control window.

Hello,

I am trying to make basic service going.
However, running into issue where the response from the service it always uninitialized response.

Here is my proto:

message ExtractWebPageContentRequest {
  string url = 1;
}

message ExtractWebPageContentResponse {
  string title = 1;
  string author = 2;
  string excerpt = 3;
  string text = 4;
  string html = 5;
  repeated Image images = 6;
  bool success = 7;
  string error = 8;
}

service ExtractionService {
  rpc ExtractWebPageContent(ExtractWebPageContentRequest) returns (ExtractWebPageContentResponse);
}

And my service:

module Extraction
  class Service < ExtractionService
    def initialize
      @extractor = Extraction::Extractor.new
    end

    def extract_web_page_content(request : ExtractWebPageContentRequest) : ExtractWebPageContentResponse
      begin
        puts "Extracting content for #{request.url}"
        raise "Something went wrong..."
        ExtractWebPageContentResponse.new
      rescue ex
        ExtractWebPageContentResponse.new success: false, error: ex.message
      end
    end
  end
end

Hello!
I'm trying to use this library for small gRPC benchmarking I have here https://github.com/LesnyRumcajs/grpc_bench

Unfortunately I noticed some issues, I'm not completely sure if they're connected to the library itself or some of it's dependencies, or it doesn't play nice with ghz defaults.

Basically manual calls work quite fine (tested with Bloom RPC), the one over ghz seem to miserably fail. All in all, there is no OK response over 30s of calls.

Summary:
  Count:	210020
  Total:	30.00 s
  Slowest:	0 ns
  Fastest:	0 ns
  Average:	6.50 ms
  Requests/sec:	7000.24

Response time histogram:

Latency distribution:

Status code distribution:
  [Unavailable]   179204 responses   
  [Unknown]       30816 responses    

Error distribution:
  [179154]   rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:50051: connect: connection refused"   
  [30816]    rpc error: code = Unknown desc = OK: HTTP status code 200; transport: missing content-type field                                                                                                                      
  [50]       rpc error: code = Unavailable desc = transport is closing                                                                                                                                                             

The same test for Ruby seems rather fine:

Summary:
  Count:	74373
  Total:	30.01 s
  Slowest:	802.53 ms
  Fastest:	0.81 ms
  Average:	20.12 ms
  Requests/sec:	2478.62

Response time histogram:
  0.810 [1]	|
  80.982 [73297]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  161.155 [32]	|
  241.327 [50]	|
  321.499 [2]	|
  401.671 [48]	|
  481.843 [0]	|
  562.015 [1]	|
  642.187 [1]	|
  722.360 [0]	|
  802.532 [97]	|

Latency distribution:
  10 % in 12.92 ms 
  25 % in 13.60 ms 
  50 % in 14.62 ms 
  75 % in 17.28 ms 
  90 % in 38.02 ms 
  95 % in 40.50 ms 
  99 % in 44.42 ms 

Status code distribution:
  [OK]                  73529 responses   
  [ResourceExhausted]   795 responses     
  [Unavailable]         49 responses      

Error distribution:
  [795]   rpc error: code = ResourceExhausted desc = No free threads in thread pool   
  [49]    rpc error: code = Unavailable desc = transport is closing                   

Same goes for other languages / libraries I have tested. Could you please take a look into this? The code for the Crystal gRPC benchmark itself is on the branch for this PR. LesnyRumcajs/grpc_bench#14

where package = 'foo.bar' (when STRIP_FROM_PACKAGE='' which is kind of strange and I think is an issue with the protobuf repo not this one)

      rpc find, receives: foo.bar.TheRequest, returns: foo.bar.TheResponse

should probably be

      rpc find, receives: Foo::Bar::TheRequest, returns: Foo::Bar::TheResponse

or a more risky approach

      rpc find, receives: TheRequest, returns: TheResponse

in addition there is the global namespace this doesn't take into account
when PROTOBUF_NS = 'Testing'
it should probably look like

     rpc find, receives: Testing::Foo::Bar::TheRequest, returns: Testing::Foo::Bar::TheResponse

I have a small application where I need to communicate over a socket on the filesystem rather than via an exposed port. It would be a great addition to be able to start a server on a Unix domain socket as an alternative to using an HTTP/2 server. I believe the core gRPC library has had this support for a while, but it hasn't filtered through to several of the other supported languages - e.g., I see no way to do this in Ruby either.

Thanks!

First of all, this is simply great! Awesome work!!

I am trying to figure out how to properly return errors on exceptions.
Any suggestions?

I just tried this which a quick ruby implementation and no messages are received by the server.

For ref: https://rubygems.org/gems/grpc/versions/1.60.0-x86_64-darwin

JSX:OSX:grpc: shards install
Resolving dependencies
Fetching https://github.com/jgaskins/grpc.git
Fetching https://github.com/jeromegn/protobuf.cr.git
Installing grpc (0.1.0 at 6503b9b2a22f3a5cebf06c08f1319b7787c1386e)
Installing protobuf (2.2.0 at d7f31df6c72218ed8c038f1337c7ef105249c38e)
Postinstall of grpc: shards build && mkdir -p ../../bin && cp bin/grpc_crystal ../../bin
Failed postinstall of grpc on shards build && mkdir -p ../../bin && cp bin/grpc_crystal ../../bin:
Resolving dependencies
Fetching https://github.com/jeromegn/protobuf.cr.git
Using protobuf (2.2.0 at d7f31df6c72218ed8c038f1337c7ef105249c38e)
Unhandled exception: Error opening file with mode 'r': '/Users/despiegk/code/github/despiegk/crystal_tests/grpc/lib/grpc/lib/protobuf/bin/protoc-gen-crystal': No such file or directory (File::NotFoundError)
  from File::new<String, String, File::Permissions, Nil, Nil>:File
  from Shards::Package#install_executables:Nil
  from Shards::Commands::Install@Shards::Command::run<String>:Nil
  from ~procProc(Array(String), Array(String), Nil)@src/cli.cr:39
  from __crystal_main
  from main

my test repo is on https://github.com/despiegk/crystal_tests/tree/master/grpc

help greatly appreciated, would be amazing to use grpc in crystal

I followed the setup instructions in the README, and everything compiles, but as soon as I run the client code with a server running (i.e. every time I fire a request), I get this from the server:

Unhandled exception in spawn: huffman string padding is larger than 7-bits (HTTP2::HPACK::Error)
  from lib/grpc/src/hpack.cr:322:17 in 'string'
  from lib/grpc/src/hpack.cr:64:21 in '->'
  from ../../../../usr/local/Cellar/crystal/0.35.1/src/primitives.cr:255:3 in 'run'

Your example repo was working great out of the box for me though, and the only difference seemed to be the lib versions, so I stepped through each release from v0.1.0 to v0.1.4 of the library, and found that when I remove the content headers by unapplying this commit, it works again, even with the changes in v0.1.4.

If there's any way I can help debug this, I'd be happy to help, though I was able to reproduce this just by creating a new project and following the README. Awesome work btw, thanks for the library.

Quick Update: Don't know what I touched just now, but now the changes in v0.1.4 are giving me a different exception... everything seems to work swell for me up until v0.1.2.