Comments (20)
Am I right that a9e31b6 is a fix for this issue?
from cbmc.
That was a first attempt. Looking at the current results, this requires more work. Currently, I see only two witnesses confirmed. I'm looking into this right now.
from cbmc.
Stripping the witness produced by
./cbmc ../sv-benchmarks/c/pthread/fib_bench_false-unreach-call.i --propertyfile ../sv-benchmarks/c/ConcurrencySafety.prp --graphml-witness witness.graphml --32
down to
seems to give something acceptable for cpachecker. I've also seen witness by other tools that just have entry and violation nodes... which is also accepted...
from cbmc.
I am currently working on witness validation of pthread-wmm/mix000_power.oepc_false-unreach-call.i
.
from cbmc.
I'll try to fix these 2 issue first:
- Missing
<data key="specification">CHECK( init(main()), LTL(G ! call(__VERIFIER_error())) )</data>
(CBMC does not write it, while ESBMC does) - Strange assumptions
<data key="assumption">\result = FALSE;</data>
from cbmc.
The specification field is inserted by the wrapper script, so it should really be there?
For the "strange" assumptions: which part of it do you consider "strange"?
from cbmc.
In my experience the strategy to solve witness problems is to manually modify the witnesses that we produce until they are accepted and only then to start implementing the changes necessary to produce the modified witness.
from cbmc.
Typical modifications that might lead to success are
- suppressing assumptions
- suppressing nodes
For example, in my example above, I removed all assumptions and kept only one node per context switch.
from cbmc.
- The "missing specification" issue is not an issue. It is present. I did not know it is generated in the wrapper script.
- The condition https://github.com/marek-trtik/cbmc/blob/sv-comp-2018-patches/src/goto-programs/graphml_witness.cpp#L164 does not for benchmarks containing variables with the
$
character. For exmaple,int y$mem_tmp;
inpthread-wmm/mix000_power.oepc_false-unreach-call.i
. I fixed that by replacing$
by#return_value
, according to description ofassumption.resultfunction
in page https://github.com/sosy-lab/sv-witnesses
from cbmc.
@peterschrammel : I applied the changes you proposed. And, for example, for pthread-wmm/mix000_power.oepc_false-unreach-call.i
we now get this result
Verification result: TRUE. No property violation found by chosen configuration.
But the correct answer should be:
Verification result: FALSE. Property violation (WitnessAutomaton) found by chosen configuration.
from cbmc.
I fixed that by replacing
$
by#return_value
Will this also work for functions that don't have a body, such as __VERIFIER_nondet_*
?
from cbmc.
@tautschnig : If you mean variables of the form return_value___VERIFIER_nondet_*$N
, like is this GOTO program:
// 168 file ../sv-benchmarks/c/pthread-wmm/mix000_power.oepc_false-unreach-call.i line 781 function P1
void *return_value___VERIFIER_nondet_pointer$1;
// 169 file ../sv-benchmarks/c/pthread-wmm/mix000_power.oepc_false-unreach-call.i line 781 function P1
return_value___VERIFIER_nondet_pointer$1 = NONDET(void *);
// 170 file ../sv-benchmarks/c/pthread-wmm/mix000_power.oepc_false-unreach-call.i line 781 function P1
P1#return_value = return_value___VERIFIER_nondet_pointer$1;
then no. I can upgrade the condition to capture these variables as well.
from cbmc.
It's ok if either of 169 or 170 results in a \result = ...
assumption in the witness. It's not necessary to cover both of them, but please do make sure that one of them ends up in there.
from cbmc.
Thanks for the notes. To be sure we do not miss anything I upgraded the condition to capture both cases.
from cbmc.
Here is the commit that provides the improved validation of our witnesses:
68fee5b
I checked the improvement on these benchmarks:
"pthread-wmm/mix000_power.oepc_false-unreach-call.i", # OK!
"pthread-wmm/mix000_power.opt_false-unreach-call.i", # OK!
"pthread-wmm/rfi008_tso.opt_false-unreach-call.i", # OK!
"pthread-lit/fkp2013_variant_false-unreach-call.i", # PROBLEM! Verification result: UNKNOWN, incomplete analysis.
"ldv-races/race-1_3-join_false-unreach-call.i", # OK!
"pthread-driver-races/char_generic_nvram_read_nvram_write_nvram_false-unreach-call.i", # OK!
"pthread-C-DAC/pthread-demo-datarace_false-unreach-call.i", # OK!
"pthread/lazy01_false-unreach-call.i", # OK!
"pthread/stack_false-unreach-call.i", # PROBLEM! Verification result: UNKNOWN, incomplete analysis. Error: Unsupported C feature (BDD-analysis does not support arrays: stack[__CPAchecker_TMP_1]) (BDDVectorCExpressionVisitor.visit, SEVERE)
"pthread-atomic/qrcu_false-unreach-call.i", # OK!
"pthread-ext/28_buggy_simple_loop1_vf_false-unreach-call.i", # PROBLEM! Verification result: UNKNOWN, incomplete analysis. Error: Unrecognized code (multiple thread assignments to same LHS not supported: t) (ThreadingTransferRelation.getNewThreadId, SEVERE)
We should re-evaluate all benchmarks before this commit is merged to the main branch.
from cbmc.
The fix/improvement of the \result
guard condition is here:
9bdb376
Please, look at it, and if you have any suspicion that it might cause troubles for some benchmark, then comment here. Thanks.
from cbmc.
Looks ok to me!
from cbmc.
We should re-evaluate all benchmarks before this commit is merged to the main branch.
Do you have some reasonable way of doing so?
from cbmc.
Yes, I am in the progress of writing a Python script which will take all the benchmarks of that category, and for each of them it will call benchexec
with 30s timeout and then CPAchecker to approve the witness. Hopefully it will work.
In this evaluation I'd like to use all today fixed we have (those to be merged to the main branch).
from cbmc.
In an attempt to rule out at least one possible cause of trouble, I've added 476e65d to output 0
and 1
instead of FALSE
/TRUE
.
Yet I'm not sure how much of a difference that makes.
from cbmc.
Related Issues (20)
- OBSOLETE
- Improve the witness generation for ProductLines, Floats, and Sequentialized
- 18 red memsafety benchmarks HOT 4
- Fixing: byte_extract flatting with negative offset: byte_extract_little_endian HOT 2
- Fixing: equality without matching types HOT 13
- Fixing: too many addressed objects: maximum number of objects is set to 2^n=256 (with n=8); HOT 2
- Fixing: too many addressed objects HOT 4
- Fixing: pointer handling for concurrency is unsound HOT 2
- Fixing: byte_update of unknown width: byte_update_little_endian HOT 2
- Fixing: implicit conversion not permitted HOT 2
- Fixing: warning: ignoring array_of HOT 3
- Fixing: cannot unpack struct with non-byte aligned components HOT 1
- RED status - strcpy/memcpy HOT 2
- Type-checking failure: "numeric exception"/"identifier XYZ not found" HOT 6
- No violation node for some benchmarks in the memory safety category HOT 3
- Incorrect FALSE detected in function strncpy (43_1a_cilled_true-unreach-call_ok_nondet_linux-43_1a-drivers--power--test_power.ko-ldv_main0_sequence_infinite_withcheck_stateful.cil.out.c) HOT 3
- Incorrect FALSE detected in function strcpy (module_get_put-drivers-char-ipmi-ipmi_watchdog.ko_true-unreach-call.cil.out.i.pp.i) HOT 6
- Incorrect FALSE related to pointer outside dynamic object bounds HOT 9
- Incorrect TRUE for warning: ignoring bswap HOT 4
- Incorrect TRUE for ldv-linux-4.0-rc1-mav/linux-4.0-rc1---drivers--usb--misc--legousbtower.ko_false-unreach-call.cil.c HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cbmc.