penberg / pstore

pstore, a high-performance, read-optimized database system.

pstore

Pstore is a high-performance, read-optimized database system. The core is implemented as a shared library libpstore that can be embedded in third-party applications.

There's also a CLI application pstore that is used to manipulate and inspect pstore files, and convert CSV files into pstore format.

The design of pstore is loosely based on C-Store which is another Open Source read-optimized database. The main idea is to store tabular data in column-order to improve read performance for special purpose workloads.

Installation

First, install dependencies:

Fedora:

$ yum install openssl-devel

Debian:

$ apt-get install libssl-dev

Then install pstore:

$ make install

The command installs an executable pstore to $HOME/bin.

Usage

The first step is to create a database file. You can import a CSV file into a pstore database with the following command:

pstore import dataset.csv dataset.out

After that, you can use the pstore cat command to look into the imported database:

pstore cat dataset.out

With pstore import --append you can append more data into a pstore database:

pstore import --append moredata.csv dataset.out

Running Tests

Pstore's test suite requires Ruby 1.9 and Bundler. If your operating system does not have Ruby 1.9, take a look at rbenv or RVM.

When you have both Ruby 1.9 and Bundler, install Rubygem dependencies:

bundle install

Then run the tests:

make check

License

Pstore is distributed under the LGPL version 2.1 license.

pstore's Issues

'libpstore' should not invoke 'die'

At the moment libpstore invokes die upon most errors, terminating the process. This severely limits its applicability elsewhere than in the pstore client.

What are your thoughts on this?

Repository layout

The repository layout in pstore is so 2012 compared to libtrading. What if it was organized as follows?

Documentation/
deps/
  fastlz/
  fields/
  snappy/
features/
include/
lib/
  compat/
scripts/
test/
tools/