hewlettpackard / nvthreads

NVthreads: Practical Persistence for Multi-threaded Applications

Authors

• Terry Hsu <[email protected]>

• Helge Bruegner <[email protected]>

• Indrajit Roy <[email protected]>

• Kimberly Keeton <[email protected]>

• Patrick Eugster <[email protected]>

Description

NVthreads is a drop-in replacement for the popular pthreads library that adds persistence to existing multi-threaded C/C++ applications. NVthreads infers consistent states via synchronization points, uses the process memory to buffer uncommitted changes, and logs writes to ensure a program’s data is recoverable even after a crash. NVthreads’ page level mechanisms result in good performance: applications that use NVthreads can be more than 2× faster than state-of-the-art systems that favor fine-grained tracking of writes. After a failure, iterative applications that use NVthreads gain speedups by resuming execution.

Master Source

https://github.com/HewlettPackard/nvthreads

Maturity

NVthreads is still under development. Please use NVthreads at your own risk. Do not deploy this research prototype to your production software before verifying the correctness and performance of your ported apps. Also, please use the master branch only, other branches are unstable research prototypes.

Dependencies

Install the following packages:

sudo apt-get install gcc-multilib
sudo apt-get install g++-multilib
sudo apt-get install libc6-dev-i386 (if you need 32-bit nvthreads)

Build & test

1. Install dummy_nvmfs: https://github.com/HewlettPackard/dummy_nvmfs

   • NOTE: For simple testing of NVthreads, this step may be skipped, so long as the path /mnt/ramdisk/nvthreads/ exists. Be aware that there will be no delays for each NVM write in this case.

2. Clone NVthreads repo:

git clone https://github.com/HewlettPackard/nvthreads

3. Create nvmfs with 1000ns delays:
   • This step is only necessary if using dummy_nvmfs from step 1.

cd $NVthreads/
./mknvmfs1000

4. Build NVthreads:

cd $NVthreads/src/
make libnvthread.so

5. Build the recovery test program:

cd $NVthreads/tests/recover/
make

6. Run the test:
   • Note: To start with a clean NVthreads environment, before beginning a test, delete the /tmp/nvlib.crash file, if it exists.

./recover_int.o  //will abort
./recover_int.o  //will recover data from previous run

Source tree structure

apps/: The applications cases for NVthreads.
    - datagen/: generates data from kmeans inputs.
    - kmeans/: implementation of the kmeans algorithm.
        - phoenix-recovery: kmeans recovery evaluation
    - pagerank/: implementation of the well-known page rank algorithm.
    - tokyocabinet-1.4.48: Tokyo Cabinet evaluation.

docs/: Published reserach paper for the NVthreads design rationale.

dummy_nvmfs: https://github.com/HewlettPackard/dummy_nvmfs

eval/: Benchmark evaluation
    datasets/: please save input data for benchmarks in this directory
    tests/: this directory contains Phoenix and PARSEC benchmarks

src/: The core of NVthreads library
    source: source code 
    include: header file

tests/: Simple test cases for NVthreads library

third-parties/: 
    atlas/: https://github.com/HewlettPackard/Atlas
    dthreads/: https://github.com/emeryberger/dthreads
    mnemosyne/: http://research.cs.wisc.edu/sonar/projects/mnemosyne/

Citing NVthreads

If you use NVthreads, please cite our reearch paper published at EuroSys 2017, included as doc/nvthreads-eurosys.pdf.

@InProceedings{nvthreads,
author = {Hsu, Terry Ching-Hsiang and Bruegner, Helge and Roy, Indrajit and Keeton, Kimberly and Eugster, Patrick},
title = {{NVthreads: Practical Persistence for Multi-threaded Applications}},
booktitle = {Proceedings of the 12th ACM European Systems Conference},
year = {2017},
series = {EuroSys 2017},
address = {New York, NY, USA},
publisher = {ACM},
doi = {10.1145/3064176.3064204},
isbn = {978-1-4503-4938-3},
location = {Belgrade, Republic of Serbia},
url = {http://dl.acm.org/citation.cfm?doid=3064176.3064204},
}

Acknowledgement

• We thank the authors of Dthreads: Tongping Liu, Charlie Curtsinger, and Emery Berger, who open sourced their software that makes NVthreads possible. Please refer to their original work at Dthreads: Efficient Deterministic Multithreading and Heap Layers: An Extensible Memory Allocation Infrastructure.
• This work was supported in part by Hewlett Packard Labs, the National Science Foundation under grant TC-1117065,
  TWC-1421910, and the European Research Council under grant FP7-617805.

nvthreads's Issues

nvthreads test run

I have build nvthreads on kernel 4.13.0-37. And tried to run the test code.
and I am getting this error. I haven't installed dummy_nvmfs, So is the reason for it.

Error:

0 : map error with addr 0xffffffffffffffff!
Aborted (core dumped)

Can this repo be made world readable?

Hello Project Contributors:

Can this github repo be made world readable?

We need to have fewer private repos in the github.com/HewlettPackard org account.

Please contact me directly at [email protected]

Thank you

xrun.h stuck waiting for token

Executing the code below has significant probability of not completing.
Code gets stuck in xrun.h, waiting for the token, with threads burning 100% CPU.

Code TL;DR: Spawn N threads. Each thread will read a counter, print its value, and increment it -- all inside a critical section.

#define NUM_THREADS 3

struct thread_args {
  int id;
}

pthread_mutex_t m;
int *counter;

void *thread_start(void *arg) {
  struct thread_args *args = (struct thread_args *) arg;
  int id = args->id;
  sleep(id);
  pthread_mutex_lock(&m);
  lprintf(">> thread %d read counter: %d\n", id, *counter);
  *counter[i]++;
  pthread_mutex_unlock(&m);
  return 0;
}

int main() {
  pthread_t tid[NUM_THREADS];
  struct thread_args args[NUM_THREADS];
  counter = (int*) nvmalloc(sizeof(int), (char*)"counter");
  if(isCrashed()) {
    nvrecover(counter, sizeof(int), (char*)"counter");
  } else {
    *counter = 0;
  }
  pthread_mutex_init(&m, NULL);
  for(int i=0; i<NUM_THREADS; i++) {
    args[i].id = i;
    pthread_create(&tid[i], NULL, thread_start, &args[i]);
  }
  for(int i=0; i<NUM_THREADS; i++) {
    pthread_join(&tid[i], NULL);
  }
  return 0;
}

Implement a simple self-referential structure using NVThreads unable to access(dereference) the pointer member variable of the structure.

Dear Author,

I am trying to implement a simple self-referential structure using NVThreads unable to access(dereference) the pointer member variable of the structure.
Say,

struct MyStruct{
int myint;
struct MyStruct * next
}

My code :
struct MyStruct* st1 = nvmalloc(size of struct, (char )''c'');
st1->myint = 1;
struct MyStruct st2 = nvmalloc(size of struct, (char *)''c'');
st2->myint = 2;
st1->next = st2;
.
.
abort here

now during recovery code is
allocate mem to st1 ;
nvrecover(st1, size of struct, (char*)"c" );
print st1->myint; --------------------->is recovered correctly;
print st1->next->myint; --------------------->gives me map error;

Please note that I dont want to nvmalloc() st2 with a new name. Because then I guess I have to maintain a list of all such nvmallocs to recover all the structures.

Please let me know if I am doing something wrong.

authorship, license

While I am delighted to see others building on our work, you really should acknowledge that Tongping Liu and I are the authors of significant portions of the code in this repo in the AUTHORS / README files; an ack of the Dthreads and Heap Layers papers would also be appreciated. In addition, that code (Heap Layers and Dthreads) was GPL v2.0 licensed; you can't re-license it with GPL v3.0 without consent, which you haven't requested and which has not been granted. Thanks.