Driver image override inside docker about bblfshd

[juanjux]

Yes, this works for me, both with the environment variable and with the --transport=docker-daemon option, but if you use the transport option without specifying an image with the env variable it will search for bblfsh/XXX-driver:latest so if you don't have an image tagged as latest (which it's not automatic) it won't find any image; I guess that's what happened to @mcarmonaa (@mcarmonaa can you confirm this?).

I'll also add that to the documentation.

[bzz]

Actually, on linux if I try

BBLFSH_DRIVER_IMAGES="python=docker-daemon:bblfsh/python-driver:dev-4dd607b;java=docker-daemon:bblfsh/java-driver:dev-45a5e8f" ./build/bblfsh server  --transport=docker-daemon

I get very similar

[error getting driver: missing driver for language java: runtime failure: Error loading image from docker engine: Error response from daemon: reference does not exist

[mcarmonaa]

@bzz I can't reproduce the last error you get in Linux

➜  build git:(master) BBLFSH_DRIVER_IMAGES="python=docker-daemon:bblfsh/python-driver:dev-4dd607b;java=docker-daemon:bblfsh/java-driver:dev-45a5e8f" bblfsh server  --transport=docker-daemon 

DEBU[0000] binding to 0.0.0.0:9432                      
DEBU[0000] initializing runtime at /tmp/bblfsh-runtime  
DEBU[0000] Overriding image for python: docker-daemon:bblfsh/python-driver:dev-4dd607b 
DEBU[0000] Overriding image for java: docker-daemon:bblfsh/java-driver:dev-45a5e8f 
DEBU[0000] setting maximum size for sending and receiving messages to 104857600 
DEBU[0000] starting server                              
DEBU[0000] registering gRPC service                     
INFO[0000] starting gRPC server                         

[bzz]

The error I reported happens not on the server startup, but when a client actually processes files

[abeaumont]

As commented offline, I think the problem is related with the transport used. docker-daemon transport looks for local images available through docker daemon (built locally), while you may expect them to get retrieved from the registry. You need to use docker transport for that. Please confirm if that's the problem and if so, we should fix and clearify the documentation.

[mcarmonaa]

In the other hand, --transport=docker-daemon is not needed since it doesn't override the transport specified for images in BBLFSH_DRIVER_IMAGES

[bzz]

That is what I tried on non-linux - make sure no Driver images exist and then

docker pull bblfsh/java-driver:dev-45a5e8f
docker pull bblfsh/python-driver:dev-4dd607b

BBLFSH_DRIVER_IMAGES="python=docker-daemon:bblfsh/python-driver:dev-4dd607b;java=docker-daemon:bblfsh/java-driver:dev-45a5e8f" docker run -e BBLFSH_DRIVER_IMAGES --privileged -p 9432:9432 --name bblfsh bblfsh/server:dev-6bc7fd7 --max-message-size=100

The result was the same - empty server logs (no requests from clients) and

[error getting driver: missing driver for language java: runtime failure: Error loading image from docker engine: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?]

[abeaumont]

Ok, running the above commands on linux I get the following client error:

> bblfsh client t.java
DEBU[0000] reading file: t.java                         
DEBU[0000] setting maximum size for sending and receiving messages to 104857600 
DEBU[0000] dialing server at localhost:9432             
DEBU[0000] instantiating service client                 
DEBU[0000] sending request                              
Status:  fatal
Errors: 
 .  error getting driver: missing driver for language java: runtime failure: Error loading image from docker engine: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
INFO[0000] exiting without error                        

[mcarmonaa]

@abeaumont @bzz I have the same error running a server in a container, but It's not related to overriding with BBLFH_DRIVER_IMAGES varibale, if you run docker run --privileged -p 9432:9432 --name bblfsh bblfsh/server:dev-6bc7fd7 --transport=docker-daemon you will get the same error.

I think it's produced because drivers are containers launched from the server container, but the server container can't see the local docker-daemon, in the end a container is isolated.

You can mount the docker.soc, docker run --privileged -v /var/run/docker.sock:/var/run/docker.sock -p 9432:9432 --name bblfsh bblfsh/server:dev-6bc7fd7 --transport=docker-daemon, to connect to the docker daemon, but the following error is obtained:

➜  build git:(master) bblfsh client foo.py
DEBU[0000] reading file: foo.py                         
DEBU[0000] setting maximum size for sending and receiving messages to 104857600 
DEBU[0000] dialing server at localhost:9432             
DEBU[0000] instantiating service client                 
DEBU[0000] sending request                              
Status:  fatal
Errors: 
 .  error getting driver: missing driver for language python: runtime failure: Error loading image from docker engine: Error response from daemon: reference does not exist
INFO[0000] exiting without error 

The server now connects to docker-daemon but it can't see the local images

:S

[abeaumont]

@mcarmonaa I think you're right, that would mean that it doesn't make sense to use the docker-daemon transport. @smola is this correct?

[juanjux]

Yes, I can reproduce this, my theory is that the server is looking for the image on the local docker daemon inside it's own docker image and not on the hosting one because the --docker-transport option works perfectly with the server running outside docker.

[juanjux]

After much code reading and messing around, there is not a programmatic solution that I know because the server inside the Docker will search for the Docker daemon that it can access, inside it's own container, but there is a very simple solution that works and it's sharing the /var/run/docker.sock Unix socket with the container:

BBLFSH_DRIVER_IMAGES="python=docker-daemon:bblfsh/python-driver:dev-4dd607b;java=docker-daemon:bblfsh/java-driver:dev-45a5e8f" docker run -e BBLFSH_DRIVER_IMAGES  -v /var/run/docker.sock:/var/run/docker.sock --privileged -p 9432:9432 --name bblfsh bblfsh/server:dev-45a5e8f

I'll modify the server documentation to reflect this.

[juanjux]

bblfsh/documentation#76

[abeaumont]

@juanjux What you propose is what @mcarmonaa already proposed earlier, but according to him, it didn't work. Did you manage to get it working?