joyrex2001 / protobuf-perlxs Goto Github PK
View Code? Open in Web Editor NEWHacked protobuf-perlxs to include rpc support for grpc-perl.
License: Apache License 2.0
Hacked protobuf-perlxs to include rpc support for grpc-perl.
License: Apache License 2.0
I have .proto files that I want to use with a mobile client talking to a
perl server. The mobile code needs to be compiled with optimize_for =
LITE_RUNTIME but the generated wrappers are wrong as they don't take this
into account as they unconditionally try to link in calls to non-lite methods.
The workaround is to preprocess the .protos in Makefile.PL to strip out the
LITE_RUNTIME for the perl target.
Would be nice to either make the perl target also understand LITE_RUNTIME.
(though I think having LITE_RUNTIME be a property of a .proto rather than a
protoc argument is probably a sub-optimal design).
Original issue reported on code.google.com by [email protected]
on 26 Oct 2009 at 7:14
What steps will reproduce the problem?
1. Use a nested message construct.
2. After compilation and use, do an ISA on the nested message and the
class is that of the nesting class, not the nested class.
What is the expected output? What do you see instead?
Here's a proto that will show the issue:
------------------
package Report;
message Statistic {
optional string key = 1;
optional string val = 2;
}
message Query {
optional string name = 1;
repeated Statistic stats = 2;
}
------------------
What version of the product are you using? On what operating system?
0.2 build on Linux 2.6.21/gcc 4.2.4/perl 5.8.8
Please provide any additional information below.
I attached a patch - simple fix. Thanks for coding this up.
-John
Original issue reported on code.google.com by [email protected]
on 4 Nov 2008 at 5:21
Attachments:
What steps will reproduce the problem?
1. ./configure --with-protobuf=/usr/local/
2. make
What is the expected output? What do you see instead?
Making all in src
g++ -g -O2 -L/usr/local/lib -o protoxs perlxs_generator.o
perlxs_helpers.o main.o -lprotoc
Undefined symbols:
"google::protobuf::FieldDescriptor::kTypeToCppTypeMap", referenced from:
(and lots, lots more undefined symbols - see attached file)
It looks like it can't find libprotoc.a but it's installed there in my
/usr/local/lib
What version of the product are you using? On what operating system?
protobuf 2.0.2, protobuf-perlxs-0.2, Mac OXS 10.5.4
Please provide any additional information below.
typescript attached showing build session
Original issue reported on code.google.com by [email protected]
on 6 Oct 2008 at 9:06
Attachments:
I noticed that strings with embedded NUL ("\0") characters in them were
being truncated when using either the HASHREF form of the ->new()
constructor or the ->set_FIELD() accessor for the generated XS class. The
problem appears to be in the conversion of the char * type to the string
type without referencing the length of the char * array. I've included a
patch that performs the necessary steps to extract the length of the char *
type from the SV pointer and use both values when constructing string
types. Please note that my experience with both XS and C++ is extremely
limited, so the patch is probably quite sub-optimal and is provided as a
proof-of-concept.
You can test this issue with something like the following:
[person.proto]
message Person {
required string name = 1;
required bytes image = 2;
}
[person.pl]
use Person;
my $p = Person->new({ name => "jason\0hord" });
$p->set_image( "binary\0goop" );
print
'name: ', $p->name, "\n",
'image: ', $p->image, "\n",
join( ' ', unpack '(H2)*', $p->pack ), "\n"
;
[pre-patch output]
name: jason
image: binary
0a 05 6a 61 73 6f 6e 12 06 62 69 6e 61 72 79
[post-patch output]
name: jason^@hord
image: binary^@goop
0a 0a 6a 61 73 6f 6e 00 68 6f 72 64 12 0b 62 69 6e 61 72 79 00 67 6f 6f 70
Original issue reported on code.google.com by [email protected]
on 21 Jan 2009 at 4:48
Attachments:
What steps will reproduce the problem?
1. Patch protobuf-perlxs-1.1/examples/embedded with the following:
Index: examples/embedded/test.pl
===================================================================
--- examples/embedded/test.pl (revision 191663)
+++ examples/embedded/test.pl (working copy)
@@ -84,6 +84,8 @@
my $packed = $error->pack;
my $length = length($packed);
my $elapsed = tv_interval($now);
+ $error->clear();
+ $error->unpack($packed);
print "Built/packed error of $length bytes in $elapsed seconds\n";
@@ -116,6 +118,8 @@
my $packed = $error->pack;
my $length = length($packed);
my $elapsed = tv_interval($now);
+ $error->clear();
+ $error->unpack($packed);
print "Built/packed error of $length bytes in $elapsed seconds\n";
2. perl Makefile.PL && make && make test
What is the expected output? What do you see instead?
I see output like this:
Built/packed error of 141 bytes in 6.8e-05 seconds
$VAR1 = {
'datetime' => 1415276412,
'severity' => 3,
'message' => 'Here iing"
test'
};
Built/packed error of 141 bytes in 6.2e-05 seconds
$VAR1 = {
'message' => 'Here iing"
test',
'datetime' => 1415276412,
'severity' => 3
};
instead of
Built/packed error of 173 bytes in 7.7e-05 seconds
$VAR1 = {
'pid' => 29231,
'message' => 'Here is a warning',
'trace' => [
{
'file' => 'test.pl',
'line' => 51
},
{
'line' => 43,
'file' => 'test.pl'
},
{
'line' => 39,
'file' => 'test.pl'
},
{
'file' => 'test.pl',
'line' => 35
},
{
'file' => 'test.pl',
'line' => 31
},
{
'line' => 27,
'file' => 'test.pl'
},
{
'line' => 23,
'file' => 'test.pl'
},
{
'line' => 19,
'file' => 'test.pl'
},
{
'file' => 'test.pl',
'line' => 15
},
{
'line' => 11,
'file' => 'test.pl'
}
],
'hostname' => 'spider-man',
'severity' => 3,
'datetime' => 1415276538
};
Built/packed error of 173 bytes in 6.1e-05 seconds
$VAR1 = {
'pid' => 29231,
'message' => 'Here is a warning',
'trace' => [
{
'file' => 'test.pl',
'line' => 51
},
{
'line' => 43,
'file' => 'test.pl'
},
{
'file' => 'test.pl',
'line' => 39
},
{
'file' => 'test.pl',
'line' => 35
},
{
'file' => 'test.pl',
'line' => 31
},
{
'file' => 'test.pl',
'line' => 27
},
{
'line' => 23,
'file' => 'test.pl'
},
{
'file' => 'test.pl',
'line' => 19
},
{
'file' => 'test.pl',
'line' => 15
},
{
'file' => 'test.pl',
'line' => 12
}
],
'datetime' => 1415276538,
'severity' => 3,
'hostname' => 'spider-man'
};
What version of the product are you using? On what operating system?
- protobuf-perlxs-1.1
- Ubuntu 14.10
- $ perl -v
This is perl 5, version 20, subversion 1 (v5.20.1) built for x86_64-linux-gnu-thread-multi
(with 37 registered patches, see perl -V for more detail)
Please provide any additional information below.
I fixed this by that patch:
Index: src/google/protobuf/compiler/perlxs/perlxs_generator.cc
===================================================================
--- src/google/protobuf/compiler/perlxs/perlxs_generator.cc (revision 191667)
+++ src/google/protobuf/compiler/perlxs/perlxs_generator.cc (working copy)
@@ -123,21 +123,19 @@
printer.Print("class $base$_OutputStream :\n"
" public google::protobuf::io::ZeroCopyOutputStream {\n"
"public:\n"
+ " explicit $base$_OutputStream(SV * sv) : sv_(sv) {}\n"
- " explicit $base$_OutputStream(SV * sv) :\n"
- " sv_(sv), len_(0) {}\n"
" ~$base$_OutputStream() {}\n"
"\n"
" bool Next(void** data, int* size)\n"
" {\n"
+ " STRLEN byteCount = SvCUR(sv_);\n"
+ " STRLEN available = SvLEN(sv_) - byteCount;\n"
+ " if (available <= 16)\n"
+ " {\n"
+ " SvGROW(sv_, 2 * byteCount);\n"
+ " available = SvLEN(sv_) - byteCount;\n"
+ " }\n"
+ " SvCUR_set(sv_, byteCount + available);\n"
+ " *data = SvPVX(sv_) + byteCount;\n"
+ " *size = available;\n"
- " STRLEN nlen = len_ << 1;\n"
- "\n"
- " if ( nlen < 16 ) nlen = 16;\n"
- " SvGROW(sv_, nlen);\n"
- " *data = SvEND(sv_) + len_;\n"
- " *size = SvLEN(sv_) - len_;\n"
- " len_ = nlen;\n"
"\n"
" return true;\n"
" }\n"
@@ -147,7 +145,11 @@
" SvCUR_set(sv_, SvLEN(sv_) - count);\n"
" }\n"
"\n"
+ " void Sync() {}\n"
- " void Sync() {\n"
- " if ( SvCUR(sv_) == 0 ) {\n"
- " SvCUR_set(sv_, len_);\n"
- " }\n"
- " }\n"
"\n"
" int64_t ByteCount() const\n"
" {\n"
@@ -156,6 +158,7 @@
"\n"
"private:\n"
" SV * sv_;\n"
- " STRLEN len_;\n"
"\n"
" GOOGLE_DISALLOW_EVIL_CONSTRUCTORS($base$_OutputStream);\n"
"};\n"
Original issue reported on code.google.com by [email protected]
on 6 Nov 2014 at 12:32
Looks like a simple typo in google/protobuf/compiler/perlxs/perlxs_generator.cc:
"discard_unkown_fields(svTHIS)\n"
Original issue reported on code.google.com by [email protected]
on 17 Jan 2011 at 1:58
Attachments:
What steps will reproduce the problem?
1. Create a protobuf message with incomplete data in ->new()
2. Call the usual message->pack()
3. libprotobuf FATAL google/protobuf/message.cc:212] CHECK failed:
What is the expected output? What do you see instead?
When I wrap the message->pack() routing inside an eval {} block, I expect
to be able to catch the exception. Instead, it seems that the exception
thrown is somehow bypassing the eval {} block.
What version of the product are you using? On what operating system?
protobuf-perlxs v0.6
perl, v5.8.8 built for i486-linux-gnu-thread-multi
Please provide any additional information below.
Here's an example code:
Using protobuf:
message Person {
required int32 id = 1;
required string name = 2;
optional string email = 3;
}
----
Perl Code:
use Person;
my $person = Person->new({id => 1});
eval {
$person->pack();
};
----
The exception should be handled. Please advise on how to gracefully handle
these exceptions from XS code. Thanks.
-- Darien
Original issue reported on code.google.com by [email protected]
on 21 Mar 2009 at 4:59
What steps will reproduce the problem?
"make test"
What is the expected output? What do you see instead?
I see: *** Signal 11
What version of the product are you using? On what operating system?
FreeBSD 9.0 using any self-compiled unthreaded perl (tried 5.8.9, 5.10.x,
5.14.2, 5.16.2) or the system perl (5.12.4, also unthreaded).
Please provide any additional information below.
With a threaded perl "make test" works.
Original issue reported on code.google.com by [email protected]
on 18 Jan 2013 at 9:59
After doing some experimentation I found an issue trying to initialize
repeated elements when passing a HASHREF to ->new() for a protoxs generated
class. I was able to track it down to an off-by-one condition in the for()
loop that initializes the repeated item in the from_hashref() helper. I've
attached a patch that solves the problem on my end.
Original issue reported on code.google.com by [email protected]
on 15 Jan 2009 at 12:55
Attachments:
What steps will reproduce the problem?
Installed for perl 5.14.1, trying example 'big':
perl Makefile.PL - ok
make - ok
make test - error:
Not a CODE reference at
/usr/local/lib/perl5/5.14.1/i686-linux-thread-multi/DynaLoader.pm line 213.
END failed--call queue aborted at blib/lib/Big/Message.pm line 213.
Compilation failed in require at test.pl line 9.
BEGIN failed--compilation aborted at test.pl line 9.
But with ubuntu's perl 5.10.1 works good:
0.017285 seconds to set 10 blob fields
0.019572 seconds to pack message of 20971570 bytes
0.022064 seconds to unpack packed message
0.045516 seconds to freeze message of 20971600 bytes
0.029589 seconds to thaw frozen hashref
What version of the product are you using? On what operating system?
Ubuntu 11.04, perl 5.14.1 (self-built).
Original issue reported on code.google.com by [email protected]
on 3 Aug 2011 at 10:49
I found this out when my message had a 'length' field, and it took me a little
while to see what was going on.
Not sure of the best way to handle this:
1) print a warning from protoxs when a message field matches a reserved method
name.
2) along with the above, create the method with a trailing undescrore (protobuf
will do this with some already, i.e. namespace_)
3) do nothing, thats just how it goes.
Original issue reported on code.google.com by [email protected]
on 17 Jan 2011 at 2:35
What steps will reproduce the problem?
1. Add next strings to examples/types/test.pl:
$msg1->add_rep_message(new Protobuf::Types::Message({ t_string => "message 1"
}));
$msg1->add_rep_message(new Protobuf::Types::Message({ t_string => "message 2"
}));
$msg1->add_rep_message(new Protobuf::Types::Message({ t_string => "message 3"
}));
$msg1->add_rep_message(new Protobuf::Types::Message({ t_string => "message 4"
}));
$msg1->add_rep_message(new Protobuf::Types::Message({ t_string => "message 5"
}));
2. Add more 'print'-s:
print Dumper $msg1->rep_message(0)->to_hashref();
print Dumper $msg1->rep_message(1)->to_hashref();
print Dumper $msg1->rep_message(2)->to_hashref();
print Dumper $msg1->rep_message(3)->to_hashref();
print Dumper $msg1->rep_message(4)->to_hashref();
3. Run 'make test'
What is the expected output? What do you see instead?
I see this:
'rep_message' => [
{
't_string' => 'message 5'
},
{
't_string' => 'message 5'
},
{
't_string' => 'message 5'
},
{
't_string' => 'message 5'
},
{
't_string' => 'message 5'
}
],
All messages are the same due to bug in to_hashref() Types.xs:
...
if ( msg0->rep_message_size() > 0 ) {
AV * av0 = newAV();
SV * sv0 = newRV_noinc((SV *)av0);
for ( int i0 = 0; i0 < msg0->rep_message_size(); i0++ ) {
::Protobuf::Types_Message * msg2 = msg0->mutable_rep_message(i0);
SV * svv1 = sv_2mortal( __Protobuf__Types_call_Method( svTHIS, "rep_message" ) );
SV * sv1 = __Protobuf__Types_call_Method( svv1, "to_hashref" );
av_push(av0, sv1);
}
hv_store(hv0, "rep_message", sizeof("rep_message") - 1, sv0, 0);
}
...
What version of the product are you using? On what operating system?
Last version and last Ubuntu.
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 13 Sep 2013 at 11:38
What steps will reproduce the problem?
1. protoxs --cpp_out=. --out=. toto.proto
2.
3.
What is the expected output? What do you see instead?
Output files ;) => infinite loop, allocate all memory
I expect output xs files, with Foo->bar that can be undef
What version of the product are you using? On what operating system?
protoxs 1.1
Please provide any additional information below.
Try protoxs with this .proto
message Foo {
repeated Foo bar = 14;
}
message OUTPUT_FORMAT {
required Foo main = 3;
}
Original issue reported on code.google.com by [email protected]
on 19 Dec 2011 at 11:12
Attachments:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.