ligurio / molly

Framework for distributed system's verification, with fault injection.

Molly

is a framework for distributed systems verification, with fault injection.

Prerequisites

• Lua interpreter: LuaJIT or LuaJIT-based is recommended.
• luafun - Lua functional library, built-in into Tarantool.
• lua-cjson - Lua library for fast JSON encoding and decoding, built-in into Tarantool.
• (optional) Jepsen-compatible consistency checker. For example elle-cli, based on Jepsen, Elle and Knossos.

Installation

• Download and setup Lua interpreter, LuaJIT or LuaJIT-based is recommended (for example Tarantool).
• Install library using LuaRocks:

$ luarocks install --local molly

NOTE: Installation of modules luafun and lua-cjson is not required when Tarantool is used, both modules are built-in there. Install them manually in case of using LuaJIT:

$ make deps-runtime

Documentation

See documentation in https://ligurio.github.io/molly/.

Examples

See also an examples in test/examples/ for SQLite database engine:

• sqlite-rw-register.lua contains a simple test that concurrently runs get and set operations on SQLite DB
• sqlite-list-append.lua contains a simple test that concurrently runs read and append operations on SQLite DB

For running examples you need installed an SQLite development package and LuaRocks.

$ sudo apt install -y sqlite3 libsqlite3-dev
$ make deps
$ make test-example

Example produces two files with history: history.txt and history.json. With elle-cli history can be checked for consistency:

$ VER=0.1.4
$ curl -O -L https://github.com/ligurio/elle-cli/releases/download/${VER}/elle-cli-bin-${VER}.zip
$ unzip elle-cli-bin-${VER}.zip
$ java -jar ./target/elle-cli-${VER}-standalone.jar -m elle-rw-register history.json
history.json        true

See tests that uses Molly library in https://github.com/ligurio/molly-tests.

Hacking

For developing molly you need to install: either LuaJIT or LuaJIT-based and LuaRocks.

$ make deps
$ export PATH=$PATH:$(luarocks path --lr-bin)
$ make check
$ make test

You are ready to make patches!

License

Copyright © 2021-2024 Sergey Bronnikov

Distributed under the ISC License.

molly's Issues

Is not parallel postgresql connection.

The code of postgresql rw-test usingmolly. After first or second call method invoike the connection was closed. It can check parallel sql command:

SELECT client_addr,pid,usename,application_name FROM pg_stat_activity;

So, it can look in output trace:

Run PostgreSQL examples with Tarantool
setup
setup
setup
setup
setup
teardown
class:	table: 0x7f7ef8691d40
close 
teardown
[INFO  2024-06-13 10:50:29:87823 ]: ERROR: test/examples/pgsql-rw-register.lua:90: calling 'execute' on bad self (LuaSQL: connection is closed)
teardown
[INFO  2024-06-13 10:50:29:87871 ]: ERROR: test/examples/pgsql-rw-register.lua:90: calling 'execute' on bad self (LuaSQL: connection is closed)
teardown
[INFO  2024-06-13 10:50:29:87889 ]: ERROR: test/examples/pgsql-rw-register.lua:90: calling 'execute' on bad self (LuaSQL: connection is closed)
teardown
[INFO  2024-06-13 10:50:29:87908 ]: ERROR: test/examples/pgsql-rw-register.lua:90: calling 'execute' on bad self (LuaSQL: connection is closed)

The close() method is not called

If we insert the print() in first position of the close() method, we will not see the output results of the method : close

sqlite_rw_register.close = function(self)
    print('close')
    self.db:close()
    return true
end

The list test is not being played

The start sqlite-list-append.lua:

	 DEV=ON /usr/bin/tarantool test/examples/sqlite-list-append.lua
	SQLite version:	3.31.1
	lsqlite3 library version:	0.9.6
	[DEBUG 2024-05-20 11:24:26:61638 ] threadpool.lua:41: Spawn a new thread 1
	[DEBUG 2024-05-20 11:24:26:61695 ] client.lua:58: Opening connection by thread 1 to DB (nil)
	[DEBUG 2024-05-20 11:24:26:61892 ] client.lua:64: Setting up DB (nil) by thread 1
	[DEBUG 2024-05-20 11:24:26:62031 ] runner.lua:69: Running test 0.000s with 1 thread(s)
	[DEBUG 2024-05-20 11:24:26:62043 ] runner.lua:74: 
	[DEBUG 2024-05-20 11:24:26:62059 ] runner.lua:79: Total planned requests: 0                                  
	[DEBUG 2024-05-20 11:24:26:62165 ] runner.lua:221: File with operations history (plain text):    history.txt
	[DEBUG 2024-05-20 11:24:26:62177 ] runner.lua:222: File with operations history (JSON):          history.json

Total planned requests: 0

SIGSEGV on running regression tests with LuaJIT

sergeyb@pony:~/sources/molly$ LUA_PATH="?/init.lua;./?.lua;/home/sergeyb/.luarocks/share/lua/5.1/?.lua;/home/sergeyb/.luarocks/share/lua/5.1/?/init.lua;
/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua" LUA_CPATH="/home/sergeyb/.luarocks/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/?.so" gdb /usr/bin/luajit test/tests.lua                                             
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2                                                                                                               
...
(gdb) run test/tests.lua
...
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7fc0540 in ?? ()
(gdb) bt
#0  0x00007ffff7fc0540 in ?? ()
#1  0x00005555555b52f7 in lj_BC_FUNCC () at buildvm_x86.dasc:809
#2  0x0000555555567a4a in gc_call_finalizer (g=g@entry=0x400003b8, L=L@entry=0x40000378, o=o@entry=0x4002e6e8, mo=<optimized out>) at lj_gc.c:475
#3  0x0000555555587ff6 in gc_finalize (L=L@entry=0x40000378) at lj_gc.c:521
#4  0x0000555555588158 in lj_gc_finalize_udata (L=<optimized out>) at lj_gc.c:528
#5  cpfinalize (L=0x40000378, dummy=<optimized out>, ud=<optimized out>) at lj_state.c:236
#6  0x00005555555b5666 in lj_vm_cpcall () at buildvm_x86.dasc:1179
#7  0x00005555555aa86e in lua_close (L=0x40000378) at lj_state.c:262
#8  0x000055555555b68e in main (argc=2, argv=<optimized out>) at luajit.c:580
(gdb) 

LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/