huntlabs / grpc-dlang Goto Github PK
View Code? Open in Web Editor NEWGrpc for D programming language, hunt-http library based.
License: Apache License 2.0
Grpc for D programming language, hunt-http library based.
License: Apache License 2.0
After successfully building the client and the server, it seems the client part is not working very well.
I have this error after the ConnectEx call in IOCP.d (I displayed this line with tracef before the first return in checkErro():
erro=-1, dwLastError=10022
The server part is working, I tried it accessing the ip and port with browser, I can see logs.
Do you know why ? thanks
Python and D examples are not able to properly talk to each other. D example client can talk to the D server, but not Python to D or D to Python. If you run the D server and Python client, the client calls SayHello and never returns but the server receives the message, just no response back. I'm not sure if the server has physically sent a response or not. If you run the Python server and the D client, the server never receives the message and the client never return from the first call to SayHello
Software versions:
$ dub build
Performing "debug" build using C:\d\dmd2\windows\bin\dmd.exe for x86_64.
grpc 0.3.0-beta.3: building configuration "library"...
source\grpc\GrpcServer.d(24,8): Error: module `core.sys.posix.signal` import `bsd_signal` not found
C:\d\dmd2\windows\bin\dmd.exe failed with exit code 1.
import helloworld.helloworld;
import helloworld.helloworldrpc;
import grpc;
import std.stdio;
auto channel = new Channel("127.0.0.1" , 50051);
GreeterClient client = new GreeterClient(channel);
HelloRequest request = new HelloRequest();
request.name = "test";
auto reply = client.SayHello(request);
if(reply !is null)
{
writeln(reply.message);
}
Hello, server running on windows or ubuntu subsystem is working well.
But when I Dockerize it, its not working anymore :
$ docker-compose up
transaction-service_app_1 is up-to-date
Attaching to transaction-service_app_1
app_1 | Fetching hunt-security 0.1.0 (getting selected version)...
app_1 | Fetching hunt-imf 0.0.7 (getting selected version)...
app_1 | Fetching boringssl 0.0.1 (getting selected version)...
app_1 | Fetching openssl 1.1.6+1.0.1g (getting selected version)...
app_1 | Fetching hunt 1.1.2 (getting selected version)...
app_1 | Fetching hunt-net 0.1.0 (getting selected version)...
app_1 | Fetching hunt-trace 0.1.9 (getting selected version)...
app_1 | Fetching protobuf 0.5.0 (getting selected version)...
app_1 | Fetching grpc 0.2.0 (getting selected version)...
app_1 | Fetching hunt-http 0.1.4 (getting selected version)...
app_1 | Performing "debug" build using dmd for x86_64.
app_1 | hunt 1.1.2: building configuration "library"...
app_1 | hunt-security 0.1.0: building configuration "library"...
app_1 | hunt-net 0.1.0: building configuration "default"...
app_1 | protobuf 0.5.0: building configuration "protobuf"...
app_1 | hunt-imf 0.0.7: building configuration "hunt-imf"...
app_1 | hunt-trace 0.1.9: building configuration "library"...
app_1 | hunt-http 0.1.4: building configuration "default"...
app_1 | grpc 0.2.0: building configuration "grpc"...
app_1 | transaction-service ~master: building configuration "application"...
app_1 | Linking...
app_1 | Running ./transaction-service
app_1 | 2019-06-04 09:31:41 (73) [debug] parseNext - after parseLine =>[pos=16 lim=411 cap=411] - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/decode/HttpParser.d:1287
app_1 | 2019-06-04 09:31:41 (73) [info] onPreface - server received preface: Http2Session@132362403889968{l:172.19.0.2:50051 <-> r:172.19.0.1:47628,sendWindow=65535,recvWindow=65535,streams=0,NOT_CLOSED} - ../../root/.dub/packages/grpc-0.2.0/grpc/source/grpc/GrpcServer.d:46
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - CtxTbl[78620610f300] decoding 255 octets - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:64
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 8683400a3a617574686f726974790f3132372e302e302e313a35303035314005... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 83400a3a617574686f726974790f3132372e302e302e313a353030353140053a... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 400a3a617574686f726974790f3132372e302e302e313a353030353140053a70... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=:authority, value=127.0.0.1:50051 - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded ':authority: 127.0.0.1:50051(preparsed h=127.0.0.1 p=50051)' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 40053a706174681c2f68656c6c6f776f726c642e477265657465722f53617948... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=:path, value=/helloworld.Greeter/SayHello - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded ':path: /helloworld.Greeter/SayHello' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 4002746508747261696c657273400c636f6e74656e742d74797065106170706c... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=te, value=trailers - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded 'te: trailers' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 400c636f6e74656e742d74797065106170706c69636174696f6e2f6772706340... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - header, name=content-type, value=application/grpc - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:176
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decoded 'content-type: application/grpc' by LitName/LitVal/Idx - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:212
app_1 | 2019-06-04 09:31:41 (73) [debug] decode - decode 400a757365722d6167656e7439677270632d6373686172702f312e32302e3120... - ../../root/.dub/packages/hunt-http-0.1.4/hunt-http/source/hunt/http/codec/http/hpack/HpackDecoder.d:76
The client (started from windows) is waiting for a response indefinitely from server running on docker container. This is the last log I have.
Just FYI:
There is another grpc-d lib:
https://github.com/dlang-community/grpc-d
and I have tried the server side, it works more reliable than this grpc-dlang library;
also it can talk to both Python and Dart client easily, e.g:
https://github.com/dlang-community/grpc-demo
https://github.com/dlang-community/grpc-d-compiler
https://github.com/dlang-community/grpc-d-interop
(I fixed a few things, so please use the dlang-community fork version of the libs.)
Basically, it's a D wrapper of Google's own libgrpc, not a home-grown library like this one, so we are using the same underlying libs from Google.
D has a relative small community, maybe we should join the effort to maintain one single quality library, instead of re-invent the wheels, unless you are confident you can do better than Google's own libs.
Cheers.
I was following README.md
$ git clone https://github.com/huntlabs/grpc-dlang
$ cd grpc-dlang
# then I was following README.md
$ dub build :protoc-gen-d
Sub package "grpc:protoc-gen-d" doesn't exist.
Finally, I figured out, what it really means is:
$ git clone https://github.com/dcarp/protobuf-d
$ cd protobuf-d
# then following README.md
$ dub build :protoc-gen-d
...
And later in the doc:
from the section "Generating protobuf code"
it should updated with the latest directory change, and just be:
$ cd examples/SimpleDemo/proto/
$ ./generate.sh
$ cd ..
$ ./build.sh
Hi,
This is related to the previous issue:
If the user set reply message to empty, the server won't send back that message, then the application client side will wait for that empty reply forever.
A request / a reply's body is empty or not is an application logic thing, the grpc library should not check for that and decide if the request need to be handled or reply sent back, the server should always do that.
Trying to use this in a project, I get a failure from hunt
:
../../../../.dub/packages/hunt-1.1.2/hunt/source/hunt/concurrency/Executors.d(401,12): Error: interface hunt.util.Common.Callable(V) at ../../../../.dub/packages/hunt-1.1.2/hunt/source/hunt/util/Common.d(120,1) conflicts with struct core.thread.context.Callable at /home/jpontaoski/dlang/dmd-2.095.0/linux/bin64/../../src/druntime/import/core/thread/context.d(26,1)
It looks like this is a very old version, as the latest one is 1.6.10
; seems that a lot of dependencies are out of date
is that supported? if not is there a timeline?
If we send a request, with empty message: e.g.
https://github.com/huntlabs/grpc-dlang/blob/master/examples/SimpleDemo/source/client.d#L15
comment out this line
// request.name = "Hunt";
Then the server side will fail this test:
grpc-dlang/source/grpc/GrpcStream.d
Line 141 in 662427b
and be trapped here forever:
grpc-dlang/source/grpc/GrpcStream.d
Line 148 in 662427b
and complains:
2022-10-17 02:24:04 | 8291 | warning | onData | The data is not ready yet. | /.dub/packages/grpc-0.5.0-beta.2/grpc/source/grpc/GrpcStream.d:148
and never pass the request to the handler.
On the client side, will see error like this:
2022-10-17 02:43:19 | 11773 | debug | remove | hunt.concurrency.ScheduledThreadPoolExecutor.ScheduledFutureTask!void.ScheduledFutureTask[Cancelled] | /.dub/packages/hunt-extra-1.2.3/hunt-
extra/source/hunt/concurrency/ScheduledThreadPoolExecutor.d:1129
You can also try using Empty message to see this bug on the server side:
google/protobuf/empty.proto
or
message Empty {}
I even tried:
message NumberMsg {
int64 value = 1; // return number
}
// on the client side:
NumberMsg req = new NumberMsg();
// but do not assign anything to req.value, the server side show the same bug.
I think we should fix this bug to handle empty message (e.g. empty string) properly.
Thanks.
This project is nice, but it does two things at the same time:
I think these two things should be decoupled, so alternative implementations of 2) can be used without changing 1). Using 2) forces ones to use a specific implementation details, i.e. logging, configuration, threading and memory management model, overload protection methods on server, load balancing methods on client, use other hunt
dependencies, etc.
First of all, thank you for creating the grpc binding. I try to build, but get a failure:
$ dub --compiler=ldc2
Fetching hunt-security 0.0.2 (getting selected version)...
Fetching boringssl 0.0.1 (getting selected version)...
Fetching openssl 1.1.6+1.0.1g (getting selected version)...
Fetching hunt 1.0.0-beta.4 (getting selected version)...
Fetching hunt-net 0.0.3 (getting selected version)...
Fetching protobuf 0.3.1 (getting selected version)...
Fetching hunt-http 0.0.4 (getting selected version)...
Performing "debug" build using ldc2 for x86_64.
hunt-grpc ~master: building configuration "hunt-grpc"...
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(106,20): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(395,20): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(396,20): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(523,16): Error: undefined identifier WSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(684,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(686,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(687,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(690,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(692,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\event\socket\iocp.d(693,5): Error: undefined identifier LPWSABUF
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(138,19): Error: function hunt.io.TcpStream.TcpStream.start does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(191,19): Error: function hunt.io.TcpStream.TcpStream.onRead does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(220,19): Error: function hunt.io.TcpStream.TcpStream.onClose does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpStream.d(240,19): Error: function hunt.io.TcpStream.TcpStream.onWrite does not override any function
..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpListener.d(122,19): Error: function hunt.io.TcpListener.TcpListener.start does not override any function ..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpListener.d(130,19): Error: function hunt.io.TcpListener.TcpListener.close does not override any function ..\..\..\AppData\Roaming\dub\packages\hunt-1.0.0-beta.4\hunt\source\hunt\io\TcpListener.d(139,29): Error: function hunt.io.TcpListener.TcpListener.onRead does not override any function
ldc2 failed with exit code 1.
Hello, I have this compilation error with latest DMD :
dub
Performing "debug" build using C:\D\dmd2\windows\bin\dmd.exe for x86.
hunt 1.0.0: target for configuration "library" is up to date.
hunt-security 0.0.6: target for configuration "library" is up to date.
hunt-net 0.0.13: target for configuration "default" is up to date.
hunt-http 0.0.13: target for configuration "default" is up to date.
protobuf 0.3.1: building configuration "protobuf"...
hunt-grpc 0.1.0: building configuration "hunt-grpc"...
..\..\..\AppData\Local\dub\packages\hunt-grpc-0.1.0\hunt-grpc\source\grpc\GrpcClient.d(3,8): Error: module `Promise` is in file 'hunt\util\concurrent\Promise.d' which cannot be read
import path[0] = ..\..\..\AppData\Local\dub\packages\hunt-grpc-0.1.0\hunt-grpc\source
import path[1] = ..\..\..\AppData\Local\dub\packages\hunt-http-0.0.13\hunt-http\source
import path[2] = ..\..\..\AppData\Local\dub\packages\hunt-1.0.0\hunt\source
import path[3] = ..\..\..\AppData\Local\dub\packages\hunt-net-0.0.13\hunt-net\source
import path[4] = ..\..\..\AppData\Local\dub\packages\hunt-security-0.0.6\hunt-security\source
import path[5] = ..\..\..\AppData\Local\dub\packages\protobuf-0.3.1\protobuf\src
import path[6] = C:\D\dmd2\windows\bin\..\..\src\phobos
import path[7] = C:\D\dmd2\windows\bin\..\..\src\druntime\import
C:\D\dmd2\windows\bin\dmd.exe failed with exit code 1.
Seems easy to fix.
EDIT : after fixing that error, there is still a lot of fix to do (hunt.container => hunt.collection, etc...).
Hi,
I'm trying the helloworld example, but I'm using a Python server, something like this:
import grpc
import HW_pb2
import HW_pb2_grpc
if __name__ == "__main__":
handler = HWHandler()
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
HW_pb2_grpc.add_HWServicer_to_server(handler, server)
server.add_insecure_port('[::]:' + str(PORT))
server.start()
And then use the D grpc client talk to it, the 1st time I run it: I got:
Thread 11 "" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffcf7fe700 (LWP 2892)]
0x0000555555aca53c in _D4hunt4http6client18Http1ClientDecoderQu6decodeMFCQBx2io10ByteBufferQmCQCs3net10ConnectionQmZv (warning: (Internal error: pc 0x555
555a69009 in read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x555555a68fa0 in read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
this=0x7fffec8589c0, warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
buffer=0x7fffeccd3e40, warning: (Internal error: pc 0x555555a69009 in read in psymtab, but not in symtab.)
session=0x7fffecd36690) at Http1ClientDecoder.d:44
44 throw new IllegalStateException("Client connection is null! The actual type is: "
The 2nd time:
grpc.GrpcException.GrpcTimeoutException@/home//.dub/packages/grpc-0.3.0-beta.4/grpc/source/grpc/GrpcStream.d(225): Timedout after 5 seconds.
----------------
??:? [0x555555d64775]
??:? [0x555555d8c9f6]
??:? [0x555555d7056d]
/home//.dub/packages/grpc-0.3.0-beta.4/grpc/source/grpc/GrpcStream.d:225 [0x55555574a6cb]
Since it's a very simple grpc server, before I investigate further, I want to ask: does this D library's grpc client can talk to a Python grpc server?
The Python side grpc code are generated with:
python -m grpc_tools.protoc -I...
with vesion:
grpcio 1.31.0
grpcio-tools 1.31.0
protobuf 3.13.0
If you know these 2 can work together, can you provide a Python grpc server example? and with the specific Python package version?
Thanks.
Please add to top level of project.
Hello,
I'm trying to write a simple grpc server in D, similar to the classic helloworld example.
I'm able to compile and start it just fine, but once I send a client request the server crashes.
The error is:
2020-07-19 18:54:13 (21) [info] onPreface - server received preface: Http2Session@132645144323280{l:127.0.0.1:50051 <-> r:127.0.0.1:33236,sendWindow=65535,recvWindow=65535,streams=0,NOT_CLOSED} - grpc-dlang/source/grpc/GrpcServer.d:64
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream: 1 - grpc-dlang/source/grpc/GrpcServer.d:69
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream headers: POST{u=http://127.0.0.1:50051/helloworld.Greeter/SayHello,HTTP/2.0,h=4,cl=-1} - grpc-dlang/source/grpc/GrpcServer.d:70
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream: 3 - grpc-dlang/source/grpc/GrpcServer.d:69
2020-07-19 18:54:13 (21) [info] onNewStream - server created new stream headers: POST{u=http://127.0.0.1:50051/helloworld.Greeter/SayHello,HTTP/2.0,h=4,cl=-1} - grpc-dlang/source/grpc/GrpcServer.d:70
Segmentation fault
I'm using the latest docker Dlang2 image to build the application and ghz
as client.
More details and the step to reproduce the crash are in this ticket: LesnyRumcajs/grpc_bench#30
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.