hartl3y94 / hsthrift

The Haskell Thrift Compiler. This is an implementation of the Thrift spec that generates code in Haskell. It depends on the fbthrift project for the implementation of the underlying transport.

The Haskell Thrift Compiler

Thrift is a serialization and remote procedure call (RPC) framework used for cross-service communication. Most services at Facebook communicate via Thrift because it provides a simple, language-agnostic protocol for communicating with structured data. Thrift can already be used in programming languages such as C++, Python, and Java using fbthrift.

The hsthrift project includes the full collection of tools and libraries for using Thrift in your own Haskell projects. The Haskell Thrift compiler generates the Haskell code needed to communicate with other Thrift services, and the included libraries allow you to build both Thrift clients and servers in Haskell. Haskell Thrift is fully compatible with all other fbthrift languages, so your Haskell project can freely communicate with other services no matter what language they are implemented in.

Note: hsthrift is compatible with fbthrift but not Apache Thrift clients and servers.

hsthrift is licensed under a standard BSD license.

Documentation

Build instructions are below.

To use Thrift, see Getting Started with Thrift in Haskell.

Cabal packages

The repository contains the following packages:

• mangle, a library for mangling C++ function prototypes into symbol names, useful for using C++ code via the Haskell FFI.
• fb-util, a collection of utilities
• thrift-compiler, the Haskell Thrift compiler
• thrift-lib, libraries for Thrift clients
• thrift-server, libraries for Thrift servers
• thrift-tests, a test suite

Building and testing

First install all dependencies (see Dependencies section below).

Then, build a local checkout of cabal-install's master branch using commit f5f8d933db229d30e6fc558f5335f0a4e85d7d44 or newer. The aforementioned commit will be available in the 3.6 release of Cabal and cabal-install.

We then use the C++ thrift compiler to generate some files needed to build the Haskell thrift compiler, which we can do immediately afterwards.

$ make thrift-cpp
$ cabal build exe:thrift-compiler

At this point, you should have a working thrift-compiler. You can check that this is indeed the case by running the following command to process many .thrift files used by tests of various packages in this repository with your freshly built compiler:

$ make thrift-hs

Finally, we can issue any cabal build/cabal run/cabal test command to build/run/test a specific component or entire package. For instance, we could test all the packages with:

$ cabal test all

# or, if there's no IPv6 configured on your machine (see below),
# you can borrow the ci configuration or derive your own:
$ cabal test --project-file=ci.cabal.project all

Source distributions

A simple

$ cabal sdist <package name>
# or, to generate all sdists
$ cabal sdist all

will generate source distribution archives under dist-newstyle/sdist/, all named following the fairly common <pkg>-<version>.tar.gz template.

The Github CI actions make use of these commands to build and test the packages from source distributions.

IPv4 vs IPv6

Some tests bring up thrift clients and servers on localhost, but designate the said localhost differently depending on whether the code is built to use ::1 (IPv6) or 127.0.0.1 (IPv4). This is determined by a tests_use_ipv4 cabal flag in thrift-lib and thrift-server. The Github CI actions turn those flags on as can be seen in the .cabal.project files at the root of this repository, because the Docker environment in which CI actions get executed does not have IPv6 configured.

Dependencies

System dependencies

Debian / Ubuntu

sudo apt install
libgoogle-glog-dev
libgflags-dev
libmysqlclient-dev
bison
flex
ninja-build
cmake
libzstd-dev

(TODO: this list is probably incomplete, please file an issue if you encounter something missing)

Fedora

sudo yum install \
     openssl-devel \
     libevent-devel \
     double-conversion-devel \
     boost-devel \
     cmake \
     ninja-build \
     glog-devel \
     gflags-devel \
     gmock-devel \
     bison \
     flex \
     libatomic \
     libsodium \
     libzstd-devel \
     pcre-devel \
     ncurses-devel \
     fmt-devel \
     gmp-devel \
     community-mysql-devel

Also you may need:

export LD_LIBRARY_PATH=/usr/local/lib:
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

Other dependencies

hsthrift also depends on:

• folly, a library of general C++ utilities
• fbthrift, Facebook's Thrift compiler and support libraries.
• Other C++ libraries required by fbthrift: rsocket-cpp, fizz, wangle.
• libfmt v8

These are typically not packaged by Linux distributions, so we have to build and install them manually from their github repos. We've provided a script in the hsthrift repository, new_install_deps.sh to do that. Run the following commands to clone the repos and build and install the dependencies (pass e.g. --threads 8 to build in parallel. Default is 4):

./new_install_deps.sh

Set your env variables to pick up the new libraries and binaries:

export LD_LIBRARY_PATH=$HOME/.hsthrift/lib:
export PKG_CONFIG_PATH=$HOME/.hsthrift/lib/pkgconfig
export PATH=$PATH:$HOME/.hsthrift/bin

make all

and test the installation with:

cabal test all