pawitp / protobuf-decoder

JavaScript-based web UI to decode ad-hoc Protobuf data

Protobuf Decoder

Protobuf Decoder is a web application based on React to decode and visualize Protobuf data (hex) into prettified table layout.

For live version, please visit: https://protobuf-decoder.netlify.app/

Getting Started

To start application locally, use npm to get the dependencies and start the application.

$ npm install
$ npm start

Once the application started, open http://localhost:3000 on web browser.

Usage

Paste Protobuf data in hex format, then press Decode button.

Example Data

Input

0a 2f 0a 08 4a 6f 68 6e 20 44 6f 65 10 01 1a 10 6a 6f 68 6e 40 65 78 61 6d 70 6c 65 2e 63 6f 6d 22 0f 0a 0b 31 31 31 2d 32 32 32 2d 33 33 33 10 01 0a 1e 0a 08 4a 61 6e 65 20 44 6f 65 10 02 1a 10 6a 61 6e 65 40 65 78 61 6d 70 6c 65 2e 63 6f 6d

Output

protobuf-decoder's Issues

SEO

Let's get this on Google!

Floats displayed with double precision

Protobuf float fields are displayed with the precision of a double due to JavaScript numbers being 64 bit. Recommended fix is to trim the number of displayed decimals to represent the actual precision of the float

Support for 64-bit integer in varint

Currently, varint decodes to JavaScript integer, causing it to not support true 64-bit integers. Varints should be decoded to JSBI.BigInt instead.

Example protobuf: 20FFFFFFFFFFFFFFFF7F

Expected result: varint: 9223372036854775807
Actual result: varint: 9223372036854776000

This tool doesn't decode url encoded ProtoBuf messages

I can't decode the following message with this tool:

!3m1!5s0x479fc1a8e70d7c1b:0x92dd4312ee42f2f8!4m22!1m10!3m9!1s0x479fc1a8e70717f3:0x316fca3a5c2c03e!2sBohemian+Hotel!5m2!4m1!1i2!8m2!3d49.0186323!4d12.091916!16s%2Fg%2F11byp8407w!3m10!1s0x479fc1a8e70717f3:0x316fca3a5c2c03e!5m2!4m1!1i2!8m2!3d49.0186323!4d12.091916!9m1!1b1!16s%2Fg%2F11byp8407w

Add export to .json option

Can you add an option to export the output into a .json file, please?

upload from file & collaboration

Hi, I made these libs (which might be helpful, here):

• rawproto
• rawprotoparse
• protoquery

I like that your tool uses other methods, so I can verify if I am doing sensible stuff in mine. We should maybe collab!

It would be helpful to be able to upload a bin-dump file (not actual upload, just use web file-api to fill the textarea.) Is this something I should PR for?

As it is, I can do this in node:

const { readFile } = require('fs/promises')
b = await readFile('test/google-play-details.bin')
console.log(b.toString('hex'))

and it works great, but it would be handy to be able to just browse for a test file I have (I have a bunch of different protobuf torture-test dumps.)

how to get the proto file from the decoded hex

Hello

Also how to get the proto file from the decoded hex using the existing code.
Also can you pls let me know finally how to convert to json format .

A bit urgent

Thanks

Please add field folding feature

Hi Pawit,

This tool is great. I am using it when debugging some protobuf issues.
Sometimes the protobuf is really big, eg 16K bytes. It's impossible to check some fields because every field is packed in a html table. I proposed to add a feature that we can click a +/- button to fold a whole field. That would be great.

Thanks

UI: Large protos get cut off

When analyzing large protos, they get cut off when they don't fit on the screen, and the X-axis scrolling is disabled because in CSS "overflow-x" is set to "hidden" for the body.

Disabling it in chrome dev tools will make it work correctly.

I'm guessing the option will need to be added as "visible" in the page's css.

Site note found, use Github CI?

Seeing "site not found" at netlify. Could this be related to my tooling PR? #87

It seems to show up fine on the preview so I am not really sure where else to look for the issue. Maybe it just needs a redeploy?

Related sidenote, I could set this up as a github action, and host on github pages, instead.

They provide free hosting, and it would link the logs in the action panel, so it might be a bit easier to troubleshoot, for people like me, that do not have access to the netlify-panel. I bet CircleCI stuff could be done in Actions, too...

Support for packed repeated field

Packed repeating fields are stored as string. We need to either automatically recognize them or provide an option for users to decode as the correct type.

See https://developers.google.com/protocol-buffers/docs/encoding#packed

Support for 64-bit integers in fixed64

Currently, fixed64 field is decoded as double. We should be able to decode it as int64 and uint64 too.

In addition, the current varint decoding may not handle int64 properly because JavaScript treats everything as a double.

how to display complete decoded hex instead of table format

Can you pls let me know how to display complete decoded hex instead of table format.

Thanks

Reading field names from .proto files

Hey! Thanks for the great and fast little app.

I am wondering, how hard would it be to add support to upload a .proto file alongside the hex string?

That way, the output could be enhanced with the field names and would be even more usable.

I would be happy to contribute to this if you could share some guidelines.

Typo in UI

Typo in UI.

Issue:
Current Caption:
Tool to decode Protobuf without having the original .proto files. All decoding is done via locally via JavaScript.

Should be fixed to:
Tool to decode Protobuf without having the original .proto files. All decoding is done locally via JavaScript.

[bug] Trying to access beyond buffer length

crack.js:1 super_copy_cracked:false
react-dom.production.min.js:4489 
        
       RangeError: Trying to access beyond buffer length
    at N (index.js:1166:36)
    at u.readInt32BE (index.js:1285:18)
    at e.value (protobufDecoder.js:30:34)
    at x (protobufDecoder.js:76:10)
    at R (ProtobufPart.js:28:19)
    at Ji (react-dom.production.min.js:3550:7)
    at Il (react-dom.production.min.js:6495:11)
    at tu (react-dom.production.min.js:5537:11)
    at Jl (react-dom.production.min.js:5526:9)
    at Vl (react-dom.production.min.js:5258:11)
Qa @ react-dom.production.min.js:4489
react-dom.production.min.js:2781 
        
       Uncaught RangeError: Trying to access beyond buffer length
    at N (index.js:1166:36)
    at u.readInt32BE (index.js:1285:18)
    at e.value (protobufDecoder.js:30:34)
    at x (protobufDecoder.js:76:10)
    at R (ProtobufPart.js:28:19)
    at Ji (react-dom.production.min.js:3550:7)
    at Il (react-dom.production.min.js:6495:11)
    at tu (react-dom.production.min.js:5537:11)
    at Jl (react-dom.production.min.js:5526:9)
    at Vl (react-dom.production.min.js:5258:11)
protobuf-decoder.netlify.app/:1 
        
       Form submission canceled because the form is not connected

Error on startup

I wanted to try out your project to get a better understanding of protobuf. I encountered an error when starting with 'yarn start'.

The console output:

Starting the development server...

Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:67:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/util/createHash.js:135:53)
    at NormalModule._initBuildHash (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:414:16)
    at handleParseError (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:467:10)
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:499:5
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:356:12
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:373:3
    at iterateNormalLoaders (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:214:10)
    at iterateNormalLoaders (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:221:10)
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:236:3
    at runSyncOrAsync (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:130:11)
    at iterateNormalLoaders (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:232:2)
    at Array.<anonymous> (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:205:4)
    at Storage.finished (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:43:16)
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:79:9
/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/react-scripts/scripts/start.js:19
  throw err;
  ^

Error: error:0308010C:digital envelope routines::unsupported
    at new Hash (node:internal/crypto/hash:67:19)
    at Object.createHash (node:crypto:133:10)
    at module.exports (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/util/createHash.js:135:53)
    at NormalModule._initBuildHash (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:414:16)
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:448:10
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/webpack/lib/NormalModule.js:322:13
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:367:11
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:233:18
    at context.callback (/home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/loader-runner/lib/LoaderRunner.js:111:13)
    at /home/aronmal/Documents/Github/protobuf-decoder-master/node_modules/babel-loader/lib/index.js:55:103 {
  opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
  library: 'digital envelope routines',
  reason: 'unsupported',
  code: 'ERR_OSSL_EVP_UNSUPPORTED'
}

Node.js v18.1.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I found a fix online.

In the package.json, change the line
"start": "react-scripts start"
to
"start": "react-scripts --openssl-legacy-provider start"

With this change, it compiled successfully.

VarInt field with INT64_MIN is decoded as INT64_MAX + 1

Schema:

message Test {
  optional int64 TestField = 1;
}

Encoded message:
08 80 80 80 80 80 80 80 80 80 01
Expected output:
As Int: -9223372036854775808
Actual output:
As Int: 9223372036854775808

This test case is also present in mapbox/protozero, which handles it as expected:
https://github.com/mapbox/protozero/blob/542fcf7dd228672b775eb283d18ccd94c4e682d9/test/t/int64/testcase.cpp#L27-L28
https://github.com/mapbox/protozero/blob/master/test/t/int64/data-min.pbf

Good job taking care of big integers! 2 online encoders that show up in Google search results give up on anything larger than 2^53, which your decoder handles correctly.

Better README

We need a README that

• Links to the production site
• Explain the goal of the project
• Include example data to play around with and a screenshot
• Explain how to run the project (yarn && yarn start)

Please add int64 support

This is a great tool for protobuf decoding. Realy like it 👍
Currently I am working on a project which uses some int64 values(As Int64 output). However it seems this tool can't decode it. Is it possible to add this support? Thanks.

[Improvement idea 💡] Add support for sint32 / sint64

Hello ! First of all, I find your project very nice and useful in educational situations!

I have a suggestion for improvement that would be useful and interesting to add. Indeed, I think it could be great, for varint variables, to display the equivalent negative value when decoding. For instance, if we have a variable of type sint32 or sint64, the value 9f a3 64 is -821456 but with standard integers, it would be 1642911.

The decoded result could present various possible values depending of the original datatype like :

int32 => 1642911
sint32 => -821456

horizontal scroll ball isn't showing in the website.

String mis-decoded as Protobuf

The string "123-456-789" is mistakenly decoded as an inner protobuf message. We should provide a way for users to override the decode format.