Fault injection parameters:
- ftruncate(2): 7% chance (triggered once)
- pread(2): 7% chance (triggered once)
- other intercepted system calls: also 7% chance, but they are triggered 0 times in this test case.
Failing test case:
5> C1.
[{[{set,{var,1},
{call,generic_qc_fsm,set_keys,
[[<<9,217,162,128,149,197,176,236,10,163,247,68,141>>]]}},
{set,{var,2},
{call,generic_qc_fsm,open,
["/tmp/generic.qc",
[read_write,
{open_timeout,0},
{max_file_size,100},
{sync_strategy,none}]]}},
{set,{var,15},
{call,generic_qc_fsm,put,[{var,2},<<"k">>,<<>>]}},
{set,{var,17},
{call,generic_qc_fsm,put,
[{var,2},
<<9,217,162,128,149,197,176,236,10,163,247,68,141>>,
<<>>]}},
{set,{var,27},{call,generic_qc_fsm,close,[{var,2}]}},
{set,{var,28},
{call,generic_qc_fsm,open,
["/tmp/generic.qc",
[read_write,
{open_timeout,0},
{max_file_size,100},
{sync_strategy,none}]]}},
{set,{var,29},{call,generic_qc_fsm,get,[{var,28},<<"k">>]}},
{set,{var,30},
{call,generic_qc_fsm,put,[{var,28},<<"k">>,<<>>]}},
{set,{var,31},{call,generic_qc_fsm,close,[{var,28}]}},
{set,{var,32},
{call,generic_qc_fsm,open,
["/tmp/generic.qc",
[read_write,
{open_timeout,0},
{max_file_size,100},
{sync_strategy,none}]]}},
{set,{var,33},
{call,generic_qc_fsm,delete,[{var,32},<<"k">>]}},
{set,{var,34},
{call,generic_qc_fsm,get,
[{var,32},<<9,217,162,128,149,197,...>>]}}],
7354},
[{verify_trace,[]}]]
The successful fault injection triggers are:
- An ftruncate(2) call returns -1, errno ENOSPC
- A pread(2) call returns -1, errno EIO
Test case trace of results:
Trace: [{set_keys,[<<9,217,162,128,149,197,176,236,10,163,247,68,141>>]},
open,
{put,yes,<<"k">>,<<>>},
{put,yes,<<9,217,162,128,149,197,176,236,10,163,247,68,141>>,<<>>},
close,open,
{get,<<"k">>,<<>>},
{put,yes,<<"k">>,<<>>},
close,open,
{delete,yes,<<"k">>},
{get,<<9,217,162,128,149,197,176,236,10,163,247,68,141>>,not_found},
close]
verify_trace: {get,expected,[<<>>],got,not_found}
verify_trace: failed
All of the DB mutations have status yes, meaning that the eleveldb API responded affirmatively to each call: i.e. there were no errors returned by the API.
This is probably not a truly minimal failing test case: the nature how the random number generator is used means that a certain number of commands needed to happen "in the middle" before a lucky 7% event happened at a crucial time. The two close & open cycles in the middle of the test case: they might be truly necessary, but they might not.
To reproduce, cut-and-paste to execute these commands. Tested with OS X. It ought to work with Linux, but honestly I haven't tested the faulterl library very much on Linux.
mkdir /tmp/e
cd /tmp/e
git clone [email protected]:basho/eleveldb.git
cd eleveldb
git checkout -f 391b885abb2005e4b3f4f96a16822c9b1f31601f
make ; rebar skip_deps=true eunit suites=NONE ; erlc -o deps/faulterl/ebin -I deps/faulterl/include priv/scenario/*erl ; deps/faulterl/ebin/make_intercept_c.escript trigger_commonpaths yo ; env `deps/faulterl/ebin/example_environment.sh $PWD/yo` erl -pz .eunit deps/*/ebin
You now have an Erlang shell running with a fault-injectable VM. Cut-and-paste to execute the following three commands.
C1 = [{[{set,{var,1},{call,generic_qc_fsm,set_keys,[[<<9,217,162,128,149,197,176,236,10,163,247,68,141>>]]}},{set,{var,2},{call,generic_qc_fsm,open,[[47,116,109,112,47,103,101,110,101,114,105,99,46,113,99],[read_write,{open_timeout,0},{max_file_size,100},{sync_strategy,none}]]}},{set,{var,15},{call,generic_qc_fsm,put,[{var,2},<<107>>,<<>>]}},{set,{var,17},{call,generic_qc_fsm,put,[{var,2},<<9,217,162,128,149,197,176,236,10,163,247,68,141>>,<<>>]}},{set,{var,27},{call,generic_qc_fsm,close,[{var,2}]}},{set,{var,28},{call,generic_qc_fsm,open,[[47,116,109,112,47,103,101,110,101,114,105,99,46,113,99],[read_write,{open_timeout,0},{max_file_size,100},{sync_strategy,none}]]}},{set,{var,29},{call,generic_qc_fsm,get,[{var,28},<<107>>]}},{set,{var,30},{call,generic_qc_fsm,put,[{var,28},<<107>>,<<>>]}},{set,{var,31},{call,generic_qc_fsm,close,[{var,28}]}},{set,{var,32},{call,generic_qc_fsm,open,[[47,116,109,112,47,103,101,110,101,114,105,99,46,113,99],[read_write,{open_timeout,0},{max_file_size,100},{sync_strategy,none}]]}},{set,{var,33},{call,generic_qc_fsm,delete,[{var,32},<<107>>]}},{set,{var,34},{call,generic_qc_fsm,get,[{var,32},<<9,217,162,128,149,197,176,236,10,163,247,68,141>>]}}],7354},[{verify_trace,[]}]].
eqc:check(eqc:testing_time(30, generic_qc_fsm:prop(false, false)), C1).
eqc:check(eqc:testing_time(30, generic_qc_fsm:prop(true, false)), C1).
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent