pjklauser / protobuf-rpc-pro Goto Github PK
View Code? Open in Web Editor NEWA java implementation for Google's Protocol Buffer RPC services.
A java implementation for Google's Protocol Buffer RPC services.
What steps will reproduce the problem?
1. the client need to pre specify a port, but it could not be got before connect
2. after connect in class DuplexTcpClientPipelineFactory, the client identifier
is filled as the predefined one, not the actual port of the channel
3. thus we could not establish multi channel between server and client
What is the expected output? What do you see instead?
my suggest change (snippet) is as follow:
---
a/src/main/java/com/googlecode/protobuf/pro/duplex/client/DuplexTcpClientPipelin
eFactory.java
+++
b/src/main/java/com/googlecode/protobuf/pro/duplex/client/DuplexTcpClientPipelin
eFactory.java
@@ -119,10 +119,11 @@ public class DuplexTcpClientPipelineFactory extends
ChannelInitializer<Channel>
}
Channel channel = connectFuture.channel();
+ InetSocketAddress address = (InetSocketAddress)
channel.localAddress();^M
ConnectRequest connectRequest = ConnectRequest.newBuilder()
.setClientHostName(clientInfo.getHostName())
- .setClientPort(clientInfo.getPort())
+ .setClientPort(address.getPort())^M
.setClientPID(clientInfo.getPid())
.setCorrelationId(correlationId.incrementAndGet())
.setCompress(isCompression())
What version of the product are you using? On what operating system?
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 14 Jan 2014 at 12:22
What steps will reproduce the problem?
1. set controller's timeout to 100 ms
2. server received the call and sleep 500 ms
3. the client wait 200 ms and check controller's status
What is the expected output? What do you see instead?
The client controller should have already changed to "failed", but the
exception is not met.
What version of the product are you using? On what operating system?
3.2.0
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 18 Jan 2014 at 3:14
I implemented Post task to protobuf-rpc-pro-streamer.
Post task is closed without response, so simple and fast.
and I thought I offer you the patch.
1. give me your eclipse(or another IDE)'s formatter of java.
2. where are you?
Original issue reported on code.google.com by [email protected]
on 3 Sep 2011 at 6:19
Protocol Buffers has been upgraded to 2.6. Would be nice if protobuf-rpc-pro
used that. Would also be nice if it would use the lite runtime instead. Don't
know if that's possible but if it is I think using that would be desirable.
Original issue reported on code.google.com by [email protected]
on 9 Sep 2014 at 9:24
also the shutdown hook should only be registered on first registration of resources, not on the constructor.
Not so much a *feature* request, but a documentation request: there are no
examples under
trunk/protobuf-rpc-pro-demo/src/main/java/com/googlecode/protobuf/pro/duplex/exa
mple/oob/ and the wiki only mentions out-of-band replies but doesn't show how
they can be used.
Original issue reported on code.google.com by [email protected]
on 11 Sep 2012 at 8:55
Support for protobuf 3.0.0. Currently protobuf-java is 3.0.0-alpha-3.1 (june 2015) when the definitive 3.0.0 version is out AND Netty supports it, we can think about upgrading.
This implementation seems to be pretty good. Any C++ implementation to use in iOS app?
I am trying to use the protoc with maven to generate the classes needed to use with your library but I am having trouble with it and I wanted to know if you were using a specific maven plugin to generate the java files from the protobuf.
Your Documentation reference a specific generated proto file but only post generation code exists.
It would be nice if the examples started with a proto file.
Version 2.5 is released:
http://protobuf.googlecode.com/svn/trunk/CHANGES.txt
From the release notes:
"Added Parser to parse directly into messages without a Builder. For
example,
Foo foo = Foo.PARSER.ParseFrom(input);
Using Parser is ~25% faster than using Builder to parse messages."
Is it possible to use Parser for a performance gain in protobuf-rpc-pro?
Original issue reported on code.google.com by [email protected]
on 6 Mar 2013 at 6:12
Hi,
I'm confused about the words 'client' and 'server' in the log output from
CategoryPerServiceLogger.
The log text below is from a server:
jun 28, 2012 11:15:43 AM com.googlecode.protobuf.pro.duplex.RpcServiceRegistry
registerService
INFO: Registered AdminRpcService
jun 28, 2012 11:15:43 AM
com.mycompany.monster.admin.service.AdminServerEngine$AdminThread run
INFO: Bound to /0:0:0:0:0:0:0:0:7999
jun 28, 2012 11:15:52 AM
com.mycompany.monster.service.AdminConnectionEventListener connectionOpened
INFO: connectionOpened RpcClientChannel->peter-pc:23[10248]
jun 28, 2012 11:15:58 AM
com.googlecode.protobuf.pro.duplex.logging.CategoryPerServiceLogger logCall
INFO: duration: -6
server: "peter-pc:23[10248]"
signature: "monster_admin_msg.AdminRpcService.GetAllStocks"
client: "peter-pc:7999[11160]"
corId: 1
request {
ts: 1340874958079
size: 29
}
response {
ts: 1340874958085
size: 19
}
In my book the output should have been:
server: "peter-pc:7999[11160]"
client: "peter-pc:23[10248]"
I other words the server and client names are swapped as I see it.
I of course understand that the word 'server' can be used both in relation to
TCP (i.e. the party which instantiates DuplexTcpServerBootstrap) as well as in
relation to the actual RCP calls (i.e. server would mean the party who responds
to RCP call) but in above scenario they are both the same, i.e. "peter-pc:7999"
is the server no matter the definition used.
I should note that when I look at log output from my client (i.e. the party
which instantiates DuplexTcpClientBootstrap) the output from
CategoryPerServiceLogger look as I expect so this is only a problem on the TCP
Server side.
Original issue reported on code.google.com by [email protected]
on 28 Jun 2012 at 12:19
What steps will reproduce the problem?
1.
Create two Out of band (OoB) message consumers:
- public class ConSumA implements RpcCallback<MessageA>
- public class ConSumB implements RpcCallback<MessageB>
- Register the OoB message consumers (RpcCallback<MessageA>) as
TcpConnectionEventListener's on the channel.
2.
ProcessA Publish MessageA on channel, then publish MessageB on channel.
channel.sendOobMessage(MessageA)
channel.sendOobMessage(MessageB)
3.
ConSumA running in ProcessB receives MessageA with expected values.
ConSumA running in ProcessB receives second MessageA with values that were sent
in MessageB. Somehow MessageB was converted to MessageA over the channel.
What is the expected output? What do you see instead?
MessageB consumed by ConSumB
What version of the product are you using? On what operating system?
<!-- protobuf & RPC dependencies -->
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<groupId>com.googlecode.protobuf-rpc-pro</groupId>
<artifactId>protobuf-rpc-pro-duplex</artifactId>
<version>3.0.4</version>
<type>jar</type>
</dependency>
Please provide any additional information below.
Apologies if this is caused by me not using the framework correctly.
Thanks for your time
Steve
Original issue reported on code.google.com by [email protected]
on 18 Sep 2013 at 9:53
Simply put, it would be great if you could get this component added to the
maven central repo
Original issue reported on code.google.com by [email protected]
on 5 Dec 2011 at 4:53
I'm trying to build an OSGi bundle using protobuf rpc, and noticed its
dependency on netty-all,
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.0.10.Final</version>
</dependency>
which is not an osgi bundle, leads to unversioned imports in the manifest. I've
tracked the relevant imports to the submodules bundles ( Import rewrapped,
[[chunks]] are mine) :
com.google.protobuf;version="[2.5,3)",
[[netty-transport]]
io.netty.bootstrap,
io.netty.channel,
[[netty-codec]]
io.netty.handler.codec,
io.netty.handler.codec.compression,
io.netty.handler.codec.protobuf,
[[netty-handler]]
io.netty.handler.ssl,
[[netty-common]]
io.netty.util.concurrent,
javax.net.ssl,
javax.security.auth.x500,org.slf4j;version="[1.7,2)"
And think switching the dependencie to the netty submodules:
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport</artifactId>
<version>4.0.10.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-common</artifactId>
<version>4.0.10.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-handler</artifactId>
<version>4.0.10.Final</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-codec</artifactId>
<version>4.0.10.Final</version>
</dependency>
will lead to a better osgi bundle, which versioned imports, avoiding problems
if several Netty versions are deployed in a container.
Original issue reported on code.google.com by [email protected]
on 20 Dec 2013 at 12:25
java.lang.StackOverflowError
at com.googlecode.protobuf.pro.duplex.client.DuplexTcpClientPipelineFactory.peerWith(DuplexTcpClientPipelineFactory.java:143)
at com.googlecode.protobuf.pro.duplex.client.DuplexTcpClientPipelineFactory.peerWith(DuplexTcpClientPipelineFactory.java:143)```
Fix is
return peerWith(host, port, null, null);
Please use Netty 3.x in branch 3.0 and Netty 4 in branch 3.1. Don't force usage
of Netty 4.
Original issue reported on code.google.com by [email protected]
on 27 Mar 2013 at 9:02
Hi I am currently using a tool to build the protobuf straight from github.
Here is the log it is saying the log4j that you are using does not work.
https://jitpack.io/com/github/pjklauser/protobuf-rpc-pro/1.1.0/build.log
Additionally when I run maven locally on my computer I get a different error:
Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-artifacts) on project protobuf-rpc-pro-duplex: Unable to execute gpg command: Error while executing process. Cannot run program "gpg": error=2, No such file or directory -> [Help 1]
It would be nice if protobuf-rpc-pro was an OSGi bundle. This can be done just
by modifying pom.xml, see
http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html
Original issue reported on code.google.com by [email protected]
on 7 Jul 2012 at 4:33
What steps will reproduce the problem?
1. Look at the google project page
What is the expected output? What do you see instead?
The source tab has the wrong SVN address.
What version of the product are you using? On what operating system?
All. All.
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 17 Dec 2012 at 8:19
I want to put this out there more as a question to users of the library....
Would you like to use protobuf-rpc-pro together with non java client libraries?
If you want straight client->server RPC with some 3rd party client, it should
be possible to support.
What C, C++, .NET, Python rpc client libs would come into question?
Original issue reported on code.google.com by [email protected]
on 8 Jan 2013 at 6:44
For some reasons, server has to handl trafic.
With ChannelTrafficShapingHandler class from netty 4.0Final
A few days ago released a new version of Netty 4.0.0Final
Original issue reported on code.google.com by [email protected]
on 18 Jul 2013 at 10:28
What steps will reproduce the problem?
1. Call
com.googlecode.protobuf.pro.duplex.RpcClient.getPipeline(RpcClient.java:447)
2. Watch the stack overflow due to infinite recursion.
What version of the product are you using? On what operating system?
3.0.1
Original issue reported on code.google.com by [email protected]
on 8 Jan 2013 at 10:37
From the little I understand JZlib's raison d'être was that java.util.zip did
not support zlib's sync flushing. This is fixed in Java 7 and has been
implemented in Netty in class JdkZlibEncoder.
Netty's compression support is pluggable to avoid unnecessary dependencies. I
like that. I would like protobuf-rpc-pro to do the same.
In the case at hand I would gladly trade a dependency for JZlib for a
dependency on Java 7 but at the moment that is not possible with
protobuf-rpc-pro.
Original issue reported on code.google.com by [email protected]
on 17 Jun 2013 at 10:43
Instead of using PIDs to check whether client/server has restarted and
requiring users to run with "-Dpid=$$", I think you could just create random
UUIDs when initializing DuplexTcpClientBootstrap/DuplexTcpServerBootstrap.
Original issue reported on code.google.com by [email protected]
on 2 Nov 2012 at 8:29
We're using your library (with great success) in a backup software we've
developed (see https://degoo.com). It's used to handle the communication
between the UI process and the background service. We're primarily using it to
update the progress of the backup and this is needs to be updated as frequently
as possible. After some performance optimizations we've done recently we've
noticed that the logging mechanism in
com.googlecode.protobuf.pro.duplex.logging.CategoryPerServiceLogger.logCall has
become a bottleneck. We've disabled the logging of this namespace in our log
config but there are still operations done that cost us some performance. Any
chance of adding some configurable if-case around this entire method so that we
can get rid of the overhead entirely?
Thanks!
Original issue reported on code.google.com by [email protected]
on 25 Oct 2014 at 2:04
https://code.google.com/p/protobuf-rpc-pro/source/browse/trunk/protobuf-rpc-pro-
duplex/src/main/java/com/googlecode/protobuf/pro/duplex/RpcServiceRegistry.java?
r=113#148
Would getFullName make more sense here and other places?
What is the expected output? What do you see instead?
I expected to be able to add various classes of the name "Service" under
different packages, but got a duplicate serviceName error.
What version of the product are you using? On what operating system?
head, macosx 10.9.1
Original issue reported on code.google.com by [email protected]
on 13 Jan 2014 at 9:20
The latest release candidate for Netty 4 breaks backwards compatibility with
previous Netty pre-releases (v4.0.0.CR1 - v4.0.0.CR4). Ouch!
See this announcement : http://netty.io/news/2013/06/18/4-0-0-CR5.html
Original issue reported on code.google.com by [email protected]
on 18 Jun 2013 at 12:26
Netty 4 brings some interresting new underlying improvements see
https://netty.io/Documentation/New+and+Noteworthy
I hope to upgrade with a 3.1.0 version of protobuf-rpc-pro after the 4.0 goes
final.
Original issue reported on code.google.com by [email protected]
on 2 Jan 2013 at 2:28
The RpcServiceRegistry cannot remove BlockingServices.
Original issue reported on code.google.com by [email protected]
on 26 May 2014 at 8:07
I am trying to run Java rpc server while python rpc client but to make it
working I need to look at your rpc implementation. It will be very beneficial
to people having most powerful protobuf rpc java implemention (No doubt in
this) and python implemention
Original issue reported on code.google.com by [email protected]
on 22 Feb 2015 at 6:37
What steps will reproduce the problem?
1. define a extension
2. register the extension to
DuplexTcpClientPipelineFactory.setExtensionRegistry() before the channel is
created.
3. received a message contain info in the extension, it is still in a vauge
binary format.
What is the expected output? What do you see instead?
What version of the product are you using? On what operating system?
3.0.7
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 10 Jan 2014 at 8:04
I need to put parts of my development into an application server.
I'm far from an expert on logging frameworks but I understand there are
classloader issues with Apache Commons Logging when the application runs inside
an application server.
This is the main reason why people use SLF4J or java.utils.logging, as far as I
understand.
Please consider to swap. SLF4J have a GUI migration tool that can be used.
Personally I would prefer to have as few external dependencies as possible,
i.e. I prefer to use JUL for logging but SLF4J is also ok as it gives people a
*choice* of log implementation at deployment time.
The request may be premature as I haven't encountered said problems myself (my
code does not yet run inside an application server) but the evidence on the net
are pretty convincing. (http://articles.qos.ch/classloader.html, don't think it
is disputed).
Original issue reported on code.google.com by [email protected]
on 5 Jul 2012 at 3:52
Hi,i am using your product in our team,
our server is some message servers,the server nodes can exchange messages
between each other by serverid,it has 5 server nodes now,the connection number
is 20, my question is if it extends to dozen of servers, how can i deal with
these connections, Or I should use some bus instead, how to gain high
performance?
appreciate your answer :) thx~
Original issue reported on code.google.com by [email protected]
on 22 Mar 2012 at 8:21
replace
RpcServerCallExecutor executor = new AsyncThreadPoolCallExecutor(3, 200);
with
RpcServerCallExecutor executor = new ThreadPoolCallExecutor(3, 200);
One small comment about your POM dependency declaration on Netty:
The excellent Netty project is no longer associated with JBoss and hence the
correct dependency declaration is now:
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
<version>3.4.1.Final</version>
</dependency>
The <groupId>org.jboss.netty</groupId> only has version of Netty prior to
version 3.2.7.
Keep up the good work.
Thank you.
Original issue reported on code.google.com by [email protected]
on 23 Apr 2012 at 6:50
I cannot get protobuf-rpc-pro to work with Java 6, and have needed to move to
Java 7. Is Java 6 actually required, or is it just that its been compiled with
the -target 1.7 that prevents its use in Java6?
Original issue reported on code.google.com by [email protected]
on 25 Jan 2014 at 6:32
This is not a bug report but a feature request / enhancement proposal. In one
of my projects I have various long running RPC operations (1-2 minutes each).
In order to have the client informed I would like to view progress bars and
percentages. I would love to see such support integrated in an RPC framework
(never seen that until now). The callback of the client could have a
progress(int percentage) method.
What do you think?
Original issue reported on code.google.com by [email protected]
on 23 May 2011 at 8:27
The cleanshutdown handler uses a single threaded ExecutorService which starts a thread ( non daemon ) which prohibits the JVM shutdown since the executor services' threadpool keeps the thread alive ( core size = 1).
a timeout mechanism has been requsted.
RpcController.setTimeout( relative timeout in millis )
and
RpcController.setDeadline( absolute time )
after which a blocking (synchronous) call is completed (as failure) - with
service exception, and a non blocking call is completed ( as failure ).
The timeout or deadline shall be transmitted to the server side and cause the
server call to be automatically cancelled when the timeout or deadline expires.
There shall be no reply to the client side after the timeout or deadline
cancellation. The client side shall monitor the timeout or deadline
independently from the server side call and terminate the call with failure by
using its own thread pool. Any server replies received after the client side
times-out shall be silently discarded ( race condition ).
There will be a change in the wireprotocol to transmit the deadline from client
to server - so anticipate a 2.1 release.
Original issue reported on code.google.com by [email protected]
on 5 Sep 2012 at 6:29
Hi, thank your for your working on this project. I am very excited about this project. Is this project stable enough to use in production environment?
Thanks & Best Regards!
The RpcClientChannel introduced "attributes" ( name value map ) in version 3.3.2. Extend the concept so that attributes can be passed in with the initial client connect ( peerWith ), so that the RpcConnectionListener events already have the attributes from the very beginning.
Hi,
I'm discovering this project, and it seems to be really powerful with some
interesting features (bidirectionnal, cancelation, ...)
Google documentation explains that "java_generic_services" option is deprecated
since v2.4 : "RPC systems based on .proto-language service definitions should
provide plugins to generate code approriate for the system."
Do you intend to switch to this new approach ?
Thanks for your project,
Original issue reported on code.google.com by [email protected]
on 1 Apr 2013 at 3:16
I was wondering if this uses Websockets underneath or if it can be changed to use websockets (as an option).
The reason being is that I would like to use it to communicate with a browser using RPC but I would like to use websockets for their low latency.
(Sorry for all of the questions!)
It would be nice if sending with compression was supported. I was able to
enable compression by adding the following to the getPipeline() function in the
channel pipeline factories:
p.addLast("deflater", new ZlibEncoder(ZlibWrapper.GZIP));
p.addLast("inflater", new ZlibDecoder(ZlibWrapper.GZIP));
However, it would be nice as an optional feature. It would also be nice if the
server could dynamically detect from the frame whether the data being sent is
compressed to support compressed and uncompressed clients simultaneously.
Original issue reported on code.google.com by [email protected]
on 17 Apr 2012 at 2:44
Support for com.google.protobuf.BlockingService would be very handfull
Original issue reported on code.google.com by [email protected]
on 28 Nov 2011 at 1:31
Hi, It would be nice to rename the WatchdogThread to a readable thread name, such as watchdog.
Thanks & Best Regards!
We have been using your wonderful library for some time now. So far, it has
been working great, with the exception of one problem. Sometimes
RpcClient.callBlockingMethod gets stuck in the while loop. What happens is that
timeToDeadline <= 0 becomes true and it tries to cancel the call but inside
RpcClient.error it gets state == null from RpcClient.removePendingRequest.
Therefore !callback.isDone() remains true indefinitely.
What steps will reproduce the problem?
I have not found a deterministic way to reproduce the error. However, it
commonly occurs after the computer returns from sleep (both the client and the
server runs on the same computer). I am guessing that it occurs due to some
race condition in how pending requests are removed.
What is the expected output? What do you see instead?
The expected output is that the method should return after the timeout.
What version of the product are you using? On what operating system?
3.0.4 on Windows 7 and Java 7.
Please provide any additional information below.
Another potential issue that I see with the code is the usage of
System.currentTimeMillis(); to determine timeouts. System.currentTimeMillis();
might return a lower value at a later point in time, if the computer clock has
changed. System.nanotime() would be more appropriate.
Original issue reported on code.google.com by [email protected]
on 30 Jul 2013 at 8:23
Hi,
I'm testing an error scenario in which a client and server have an open
connection, the server dies and the client tries to use the connection. I get a
infinite loop that I can locate in the do...while in RpcClient.handleClosure():
The problem is that the while condition checks that the pendingRequestMap.size
> 0, but in the do{} only the keys that were available before the do{} are
iterated over. So if a new key is added after the iteration starts, that key
will never be handled, and the while condition will always return true. I
suggest the line pendingCallIds.addAll(pendingRequestMap.keySet()); be moved
inside the do{}.
I'm using version 1.2.0.
Br,
-M-
Original issue reported on code.google.com by [email protected]
on 11 Jun 2012 at 9:59
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.