Question about performance about bblfshd

[juanjux]

Some ideas for that OSD:

• Benchmark also against connecting directly to the driver via gRPC (just run the driver image and connect to it, it speaks the same protocol as the server). This will give the load bblfshd adds.

• Benchmark the Python, Java and Scala drivers with files of more or less the same complexity.

• Hack the native drivers a little for them do to their parse in a loop without any gRPC calling.

• Compare the speed of Parse vs ParseNative. This would give the load added by the annotating process.

I can help you with this that OSD day.

[juanjux]

The native parse is implemented in pure C. The non native parse must do a second pass to add missing positions (the ones we can), comments, et cetera, so it's much slower. Then you need to add the protocol round trip, once for the client to the server and another one from the server to the driver... you get the picture. Still, at some point we should need to benchmark every part and find fixable bottlenecks.

[smacker]

Thank you for replay. I perfectly understand why it’s slower.
My question is: do you have performance improvements in roadmap? If not, will you be open to pull requests with profiling/benchmarks/optimizations?

I’m sorry if it wasn’t clear from my first message.

[mcuadros]

We never get focused on performance, the last improvements we did were focused on resilience.

Said this, as @juanjux explained, what you are obtaining from the python AST is not the same information you get from the bblfshd server. Also, you executed this on linux or macOS?

Also, these metrics are for Python, so maybe is the driver, maybe is the server, etc. Definitely, we should get some profiling stats, to at least know how many time takes each part.i

For sure the performance can be improved and we should improve, but at least in the latest metrics I got from engine, bblfshd is not being the bottleneck at all.

[smacker]

Okay. Not in roadmap. Thanks!

Let me give more context. I'm working on hercules-web during OSDs. I want to calculate function hotness. But it's way too slow for web service even for small repos.
I wanted to spend my next OSDs writing some benchmarks and try to optimize it.

But very often performance optimizations add complexity to code. Can I expect such PRs to be merged or if it's totally not a focus will it be rejected?

[juanjux]

Out of focus PRs won't be rejected, the focus is for the language-analysis team planned tasks/epics and doesn't apply to contributions from other teams or OSDs.

[smacker]

Thanks a lot! My own thoughts how to do it are very similar :)
I just don't think I need to hack the native drivers. Every driver has cli tool which reads from stdin. I can use it!

[smacker]

got some results: https://github.com/smacker/bblfsh-benchmark