dolftax / bblfshd

A self-hosted server for source code parsing

bblfshd

This repository contains bblfsh daemon (bblfshd), which includes the runtime that runs the driver in containers and the bblfshctl, a cli tool used too control the installed drivers and query the status of the daemon.

Drivers are implemented as docker image that each have their own repository in the bblfsh organization on GitHub. For more information, see bblfsh SDK documentation.

Getting Started

See the Getting Started guide.

Quick start

The recommended way to run bblfshd is using docker:

docker run -d --name bblfshd --privileged -p 9432:9432 -v /var/lib/bblfshd:/var/lib/bblfshd bblfsh/bblfshd

On macOS, use this command instead to use a docker volume:

docker run -d --name bblfshd --privileged -p 9432:9432 -v bblfsh-storage:/var/lib/bblfshd bblfsh/bblfshd

The container should be executed with the --privileged flag since bblfshd it's based on container technology and interacts with the kernel at a low level. bblfshd, expose a gRPC server at the port 9432 by default, this gRPC will be used by the clients to interact with the server. Also, we mount the path /var/lib/bblfshd/ where all the driver images and container instances will be stored.

Now you need to install the driver images into the daemon, you can install the official images just running the command:

docker exec -it bblfshd bblfshctl driver install --all

You can check the installed versions executing:

docker exec -it bblfshd bblfshctl driver list

+----------+-------------------------------+---------+--------+---------+--------+-----+-------------+
| LANGUAGE |             IMAGE             | VERSION | STATUS | CREATED |   OS   | GO  |   NATIVE    |
+----------+-------------------------------+---------+--------+---------+--------+-----+-------------+
| python   | //bblfsh/python-driver:latest | v1.1.5  | beta   | 4 days  | alpine | 1.8 | 3.6.2       |
| java     | //bblfsh/java-driver:latest   | v1.1.0  | alpha  | 6 days  | alpine | 1.8 | 8.131.11-r2 |
+----------+-------------------------------+---------+--------+---------+--------+-----+-------------+

To test the driver you can executed a parse request to the server with the bblfshctl parse command, and an example contained in the docker image:

docker exec -it bblfshd bblfshctl parse /opt/bblfsh/etc/examples/python.py

SELinux

If your system has SELinux enabled (is the default in Fedora, Red Hat, CentOS and many others) you need to compile and load a policy module before running the bblfshd Docker image or running driver containers will fail with a permission denied message in the logs.

To do this, run these commands from the project root:

cd selinux/
sh compile.sh
semodule -i bblfshd.pp

If you were already running an instance of bblfshd, you will need to delete the container (docker rm -f bblfshd) and run it again (docker run...).

Once the module has been loaded with semodule the change should persist even if you reboot. If you want to permanently remove this module run semodule -d bblfshd.

Alternatively, you could set SELinux to permissive module with:

echo 1 > /sys/fs/selinux/enforce

(doing this on production systems which usually have SELinux enabled by default should be strongly discouraged).

Development

If you wish to work on bblfshd , you'll first need Go installed on your machine (version 1.9+ is required) and Docker, docker its used to build and run the test in an isolated environment.

For local development of bblfshd, first make sure Go is properly installed and that a GOPATH has been set. You will also need to add $GOPATH/bin to your $PATH.

Next, using Git, clone this repository into $GOPATH/src/github.com/bblfsh/bblfshd. All the necessary dependencies are automatically installed, so you just need to type make. This will compile the code and then run the tests. If this exits with exit status 0, then everything is working!

Dependencies

Ensure you have ostree and development libraries for ostree installed.

You can install from your distribution pack manager as follow, or built from source (more on that here).

Debian, Ubuntu, and related distributions:

$ apt-get install libostree-dev

Fedora, CentOS, RHEL, and related distributions:

$ yum install -y ostree-devel

Arch and related distributions:

$ pacman -S ostree

Building From Source

Build with:

$ make build

Running Tests

Run tests with:

$ make test

License

GPLv3, see LICENSE