Comments (10)
Fixed the cause of all tests I have that resulted in container hangs: bblfsh/sdk/pull/135
We'll test together on monday and see if we can close it.
The timeout and the container killing will be done in a separate PR since I want to check some things with my team.
from bblfshd.
@EgorBu and I just checked that this (already merged) fix and the previous ones make the previous problems unreproducible. We banged the server hard with three processes sending it all the tensorflow .py
files, repeated 20 times, and the files are always the same and the server didn't hang.
I'll release new docker images of the server and the Python & Java drivers today or tomorrow with all these fixes incorporated. Closing the issue, if the new docker images doesn't work we'll reopen it.
from bblfshd.
@vmarkovtsev Can you explain which command you use to reproduce the bug?
from bblfshd.
I've found some instances of the python-client hanging randomly on my debugging of #34. It seems to happen more frequently after the previous request failed for any reason and @smola and I suspect the driver containers could not be closing correctly after some errors, which could cause all the provisioned containers to be in a non-responsive state at the end.
I'll look into it as part of that bug but maybe the fix could be the same for this one. I will post an update when I've more information.
from bblfshd.
I'll be fixing bblfsh/python-driver/issues/27 first since it could also be related (the problem seem to be worsened by driver failures).
from bblfshd.
Update: now that I've the docker to test in a controlled environment I'm seeing different behaviours on different tests:
-
I can bang the server as much as I want in paralell (even to the point of killing my machine swapping) with a Python file that produces quickly a parse error and the server never fails in this way. I think the fix to #34 helped with this. Also tried with other kinds of errors produced by the native driver with the same result.
-
...but if I send a request that produces the buffer size error from the SDK (it must be a really huge one now that the buffer size is 4MB) after the first request the server doesn't reply anymore. I'm investigating this.
from bblfshd.
Update:
This bug have shown some existing problems that need to be fixed.
Currently there is a problem that show with some files that produce an error in the Go part of the driver (the SDK) where an error is correctly returned but the next request after that one hangs. That's because the driver reads from the driver container stdin but doesn't write anything to stdout, so the server to driver communication hangs (and thus also the client to server one) .
This doesn't mean that the server hangs; other connections to it while the first one is waiting will work since the server will instantiate more containers to manage these new connections.
- Modify the encoding and decoding logic in the SDK so request to hanged drivers can timeout.
- When a request timeouts trying to read or write from a driver container, log the problem and kill it.
- Discover why a driver hangs on the second request after my torture files (even while the error is correctly returned on the first one) and fix it.
from bblfshd.
PS: @bzz
from bblfshd.
here it's code that we used to test
- repository that gives different uasts for different tries: https://github.com/tensorflow/tensorflow
because of JSON decoder buffer size - if it exceeds the limit than the driver is broken and returns errors for future requests -> so it make uasts generation not reproducible. @juanjo told that itβs related to :
bblfsh/sdk#135
bblfsh/sdk#130
bblfsh/python-driver#26
and should be published in new docker image (today?) - code to reproduce:
# download scripts https://gist.github.com/EgorBu/dafee1247e91a8faee6328a19e67cad7
git clone https://github.com/tensorflow/tensorflow
cd tensorflow
# it may take a lot of time
bash ../test_34_36_tensorflow.sh
# if it's broken - here it's python script to find the first difference in files
# https://gist.github.com/EgorBu/f17353e5ed5c63f7afb865c8fad7f8c8
python finddiff.py
# don't forget to delete tmp files with uasts
from bblfshd.
New Python, Java and Server docker images have been released with all the latest fixes.
from bblfshd.
Related Issues (20)
- Refactor scalability mechanism - make it _container agnostic_ HOT 6
- install --all has the same effect as --recommended HOT 1
- bblfshd and go-client interdependency HOT 1
- Design/Prototype scalability and deployment support
- Design/Prototype local scalable environment
- Prototype native parser scaling mechanism HOT 3
- Test scalability layers
- Drivers discovery mechanism HOT 2
- make test fails HOT 2
- cron job fails in Travis
- Is this still active? HOT 1
- Archive project(s)
- Ongoing maintenance of the Babelfish project HOT 6
- Driver instance not killed after deadline
- Cannot get latest version: module contains a go.mod file, so module path should be github.com/bblfsh/bblfshd/v2 HOT 5
- Security Issue: Request for contact
- bblfshd not run HOT 1
- https://doc.bblf.sh/ down?
- Babelfish run test error
- Is bblfshd dead? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bblfshd.