cisco-talos / moflow

Release Branches for MoFlow

moflow-0.7

First checkin includes tools based on BAP framework

To get started, install dependencies listed in the BAP-0.7-moflow/INSTALL file then run the build_everything.sh script from the BAP-0.7-moflow directory. This version of moflow was written and tested on Ubuntu 12.04 LTS

Utilities can be found in the custom_utils directory

Current tools include:

slicer - post-crash graph backtaint slicer

motriage - post-crash forward symbolic emulator looking for more exploitable conditions

egas - moflow clone of SAGE

The pintool also includes substantial changes compared to the original

More tracing and debugging tools will be released shortly

moflow's Issues

Generated binary cannot load libdyninstAPI_RT.so and libAflDyninst.so [fixed]

I followed the instructions to instrument my binary successfully, but it complained when I tried to run the program:

[zhangys@chcpu12a afl-dyninst]$ export DYNINSTAPI_RT_LIB=/home/zhangys/local/lib/libdyninstAPI_RT.so

[zhangys@chcpu12a afl-dyninst]$ ./afl-dyninst -i ../../benchmarks/binary/lavam_binary/base64 -o base64_inst
Skipping library: libAflDyninst.so
Skipping library: libAflDyninst.cpp
Instrumenting module: base64
Skipping library: ld-linux-x86-64.so.2
Skipping library: libc.so.6
Inserting init callback.
Saving the instrumented binary to base64_inst...
All done! Happy fuzzing!

[zhangys@chcpu12a afl-dyninst]$ ./base64_inst 
./base64_inst: error while loading shared libraries: libdyninstAPI_RT.so: cannot open shared object file: No such file or directory

[zhangys@chcpu12a afl-dyninst]$ ./base64_inst 
./base64_inst: error while loading shared libraries: libAflDyninst.so: cannot open shared object file: No such file or directory

[zhangys@chcpu12a afl-dyninst]$ ls
afl-dyninst      afl-dyninst.o  base64_inst        libAflDyninst.so  Makefile  README.txt
afl-dyninst.cpp  AUTHORS        libAflDyninst.cpp  LICENSE           rand.b64

I found that, I need to:

• put export DYNINSTAPI_RT_LIB=xxx into ~/.bashrc
• put libAflDyninst.so to environment variable LD_LIBRARY_PATH, fixed the second one

export DYNINSTAPI_RT_LIB=/home/zhangys/local/lib/libdyninstAPI_RT.so

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/zhangys/fuzzing/moflow/afl-dyninst

afl-dyninst: Segmentation fault when instrumenting Position Independent Executable

If you try to instrument an Position Independent Executable (code compiled with the "-fPIC -pie" options for exemple), afl-dyninst segfault.

Here is what I found:
On line 250, defaultModule is declared.
On line 265 defaultModule get its value. But this line is only executed if there is a module named DEFAULT_MODULE. If the executable is a PIE, dyninst doesn't produce a module named DEFAULT_MODULE.
Then, line 289 is executed. But because line 265 is never executed, defaultModule is still equal to NULL.

Error getting shm and Segmentation fault

Program crashes itself (tried with -s):

[zhangys@chcpu12a afl-dyninst]$ ./afl-dyninst -i ../../benchmarks/binary/lavam_binary/base64 -o base64_inst -s 300
Skipping library: libAflDyninst.cpp
Skipping library: libAflDyninst.so
Instrumenting module: base64
Skipping library: ld-linux-x86-64.so.2
Skipping library: libc.so.6
Inserting init callback.
Saving the instrumented binary to base64_inst...
All done! Happy fuzzing!

[zhangys@chcpu12a afl-dyninst]$ ./base64_inst -d ../../lavam/base64/fuzzer_input/rand.b64 
Error getting shm
Segmentation fault

[zhangys@chcpu12a afl-dyninst]$ ./afl-dyninst -i ../../benchmarks/binary/lavam_binary/base64 -o base64_inst -s 500
Skipping library: libAflDyninst.cpp
Skipping library: libAflDyninst.so
Instrumenting module: base64
Skipping library: ld-linux-x86-64.so.2
Skipping library: libc.so.6
Inserting init callback.
Saving the instrumented binary to base64_inst...
All done! Happy fuzzing!

[zhangys@chcpu12a afl-dyninst]$ ./base64_inst -d ../../lavam/base64/fuzzer_input/rand.b64 
Error getting shm
Segmentation fault

It also crashes when using afl-fuzz:

[zhangys@chcpu12a afl]$ ./afl-fuzz -i ../../lavam/base64/fuzzer_input/ -o ../../output/base64_dyn -m none -- ./lavam/base64_inst -d @@
afl-fuzz 2.52b by <[email protected]>
[+] You have 80 CPU cores and 5 runnable tasks (utilization: 6%).
[+] Try parallel jobs - see docs/parallel_fuzzing.txt.
[*] Checking CPU core loadout...
[+] Found a free CPU core, binding to #4.
[*] Checking core_pattern...
[*] Checking CPU scaling governor...
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Scanning '../../lavam/base64/fuzzer_input/'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Attempting dry run with 'id:000000,orig:rand.b64'...
[*] Spinning up the fork server...
[+] All right - fork server is up.

[-] Oops, the program crashed with one of the test cases provided. There are
    several possible explanations:

    - The test case causes known crashes under normal working conditions. If
      so, please remove it. The fuzzer should be seeded with interesting
      inputs - but not ones that cause an outright crash.

    - Least likely, there is a horrible bug in the fuzzer. If other options
      fail, poke <[email protected]> for troubleshooting tips.

[-] PROGRAM ABORT : Test case 'id:000000,orig:rand.b64' results in a crash
         Location : perform_dry_run(), afl-fuzz.c:2852

Failed to open instrumentation library?

Hi, I have (finally) built Dyninst and got afl-dyninst to build as well. Like the git instructions say, my libAflDyninst.so is in my current directory but after I run afl-dyninst, it errors out on loading the instrumentation library (it errors saying "The instrumentation library must be in the current working directory."). Is it possible it's having an issue because the binary I am trying to load is a 32 bit binary on a 64 bit processor? This particular application distributes only 32 bit binaries and requires the user to have ia32libs type package on their distribution to run it.

A question about gentrace

When I use gentrace.so to build trace.bpt , it show error.

terminate called after throwing an instance of 'SerializedTrace::TraceException'
what(): Attempt to add zero-length frame to trace
Aborted (core dumped)

I don't know why?

Instrumenting PrinceXML blackbox binary results in "Error getting shm"

Using afl-dyninst to instrument the PrinceXML blackbox linux binary. After instrumenting, I get "Error getting shm" unless I am using afl-showmap, which prints "Error writing fork server".

Running afl-fuzz with the instrumented binary yields no new paths found, and I am assuming it is hitting the Error getting shm as well, but I can't be certain at this point.

Running afl-fuzz -Q works with the plain blackbox binary.

Any thoughts? I've attempted skipping up to 5000 basic blocks for instrumentation but it persists.

Full output of what I did below:

root@w00den-fuzzer:~/prince-10r7-ubuntu16.04-amd64/lib/prince# LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH ~/moflow/afl-dyninst/afl-dyninst -i bin/prince -o bin/prince_instr
Skipping library: libAflDyninst.so
Instrumenting module: DEFAULT_MODULE
Skipping library: ld-linux-x86-64.so.2
Skipping library: libc.so.6
Skipping library: libcom_err.so.2
Skipping library: libcrypt.so.1
Skipping library: libcrypto.so.1.0.0
Skipping library: libdl.so.2
Skipping library: libexpat.so.1
Skipping library: libgcc_s.so.1
Skipping library: libkeyutils.so.1
Skipping library: liblzma.so.5
Skipping library: libm.so.6
Skipping library: libpng12.so.0
Skipping library: libpthread.so.0
Skipping library: libresolv.so.2
Skipping library: libssl.so.1.0.0
Skipping library: libz.so.1
Skipping library: libasn1.so.8
Skipping library: libcurl.so.4
Skipping library: libffi.so.6
Skipping library: libfontconfig.so.1
Skipping library: libfreetype.so.6
Skipping library: libgif.so.7
Skipping library: libgmp.so.10
Skipping library: libgnutls.so.30
Skipping library: libgssapi.so.3
Skipping library: libgssapi_krb5.so.2
Skipping library: libhcrypto.so.4
Skipping library: libheimbase.so.1
Skipping library: libheimntlm.so.0
Skipping library: libhogweed.so.4
Skipping library: libhx509.so.5
Skipping library: libicudata.so.55
Skipping library: libicuuc.so.55
Skipping library: libidn.so.11
Skipping library: libjbig.so.0
Skipping library: libjpeg.so.8
Skipping library: libk5crypto.so.3
Skipping library: libkrb5.so.26
Skipping library: libkrb5.so.3
Skipping library: libkrb5support.so.0
Skipping library: liblber-2.4.so.2
Skipping library: libldap_r-2.4.so.2
Skipping library: libnettle.so.6
Skipping library: libp11-kit.so.0
Skipping library: libroken.so.18
Skipping library: librtmp.so.1
Skipping library: libsasl2.so.2
Skipping library: libsqlite3.so.0
Skipping library: libstdc++.so.6
Skipping library: libtasn1.so.6
Skipping library: libtiff.so.5
Skipping library: libwind.so.0
Skipping library: libxml2.so.2
Inserting init callback.
Saving the instrumented binary to bin/prince_instr...
All done! Happy fuzzing!
root@w00den-fuzzer:~/prince-10r7-ubuntu16.04-amd64/lib/prince# bin/prince_instr 
Error getting shm
Usage:
  prince [OPTIONS] doc.html              Convert doc.html to doc.pdf
  prince [OPTIONS] doc.html -o out.pdf   Convert doc.html to out.pdf
  prince [OPTIONS] FILES... -o out.pdf   Combine multiple files to out.pdf

Try 'prince --help' for more information.

root@w00den-fuzzer:~/prince-10r7-ubuntu16.04-amd64/lib/prince# afl-showmap -o /dev/null -- bin/prince_instr 
afl-showmap 2.35b by <[email protected]>
[*] Executing 'bin/prince_instr'...

-- Program output begins --
bin/prince_instr: error while loading shared libraries: libc.so.6: failed to map segment from shared object
-- Program output ends --

[-] PROGRAM ABORT : No instrumentation detected
         Location : main(), afl-showmap.c:718

root@w00den-fuzzer:~/prince-10r7-ubuntu16.04-amd64/lib/prince# afl-showmap -o /dev/null -m1000 -- bin/prince_instr 
afl-showmap 2.35b by <[email protected]>
[*] Executing 'bin/prince_instr'...

-- Program output begins --
Error writting fork server
Usage:
  prince [OPTIONS] doc.html              Convert doc.html to doc.pdf
  prince [OPTIONS] doc.html -o out.pdf   Convert doc.html to out.pdf
  prince [OPTIONS] FILES... -o out.pdf   Combine multiple files to out.pdf

Try 'prince --help' for more information.

-- Program output ends --
[+] Captured 1144 tuples in '/dev/null'.
root@w00den-fuzzer:~/prince-10r7-ubuntu16.04-amd64/lib/prince# 

llvm_codegen.ml Unbound module during make

hi,
i'm using ubuntu 15 and have sucecssfully installed all needed dependedcies (llvm-3.6) and the configure was ok as well (this wasn't easy.. :-) )

during the make i'm getting issue:
Unbound module initialize_native_target in llvm_codegen.ml (line 55)

since i have seen this is an "ignor" i thoght i will just remove this like but than i got again another issue
with executionengine that is now Unbound ..

see snip of the make:

Making all in ocaml
make[1]: Entering directory '/home/toor/moflow/BAP-0.7-moflow/ocaml'
make[2]: Entering directory '/home/toor/moflow/BAP-0.7-moflow/ocaml'
make[2]: 'libbap_stubs.a' is up to date.
ocamlfind ocamlc -package bigarray,str,num,unix,camomile,threads,piqi.lib,llvm,llvm.analysis,llvm.executionengine,llvm.target,llvm.scalar_opts,llvm.bitwriter -c -g -thread -warn-error Aelz -annot -I /usr/lib/ocaml/camlp4 -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../zarith-1.0 -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../batteries/_build/src -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../ounit/_build/src/ -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../pcre-ocaml/lib -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../ocamlgraph-1.8 -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../objsize-0.16 -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../libtracewrap/libtrace/src/ocaml -I /home/toor/moflow/BAP-0.7-moflow/ocaml/../libtracewrap/libtrace/src/ocaml/piqi -I piqi/ llvm_codegen.ml
File "llvm_codegen.ml", line 61, characters 19-41:
Error: Unbound module ExecutionEngine
OCamlMakefile:1057: recipe for target 'llvm_codegen.cmo' failed
make[2]: *** [llvm_codegen.cmo] Error 2
make[2]: Leaving directory '/home/toor/moflow/BAP-0.7-moflow/ocaml'
OCamlMakefile:851: recipe for target 'debug-code-library' failed
make[1]: *** [debug-code-library] Error 2
make[1]: Leaving directory '/home/toor/moflow/BAP-0.7-moflow/ocaml'
Makefile:286: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

would be nice if you can support in docker image so this installation process will be not needed.. take too long to get the exact versioning like you had ..

Thanks

BAP "no thread id exists" error

After my compilation fixes (see the pull request I sent you), I get the following:

$ ./prep-slice.sh ./tests/out.bpt ./tests/out.il
Fatal error: exception Failure("Can not lookup vars if no thread id exists!")

$ ./prep-slice.sh ./demo/out.bpt ./demo/out.il
Fatal error: exception Failure("Can not lookup vars if no thread id exists!")

The trace file is generated in both cases with the corresponding trace.sh wrapper.

Installing is a pain

Every time I install afl-dyninst, it takes a few hours as I need to set up Dyninst and many supporting libraries (libdwarf, libiberty, etc) by hand.

The worst part is ensuring the libraries expected are the correct version.

Some documentation on the expected library versions (dyninst 8.2 or 8.1?) and expected OS setup (ubuntu tested, etc) would greatly help.