lunixbochs / usercorn

dynamic binary analysis via platform emulation

usercorn

Building

Usercorn depends on Go 1.6 or newer, as well as the latest unstable versions of Capstone, Unicorn, and Keystone.

make deps (requires cmake) will attempt to install all of the above dependencies into the source tree under deps/.

make will update Go packages and build usercorn

Example Commands

usercorn run bins/x86.linux.elf
usercorn run bins/x86_64.linux.elf
usercorn run bins/x86.darwin.macho
usercorn run bins/x86_64.darwin.macho
usercorn run bins/x86.linux.cgc
usercorn run bins/mipsel.linux.elf

usercorn run -trace bins/x86.linux.elf
usercorn run -trace -to trace.uc bins/x86.linux.elf
usercorn trace -pretty trace.uc
usercorn run -repl bins/x86.linux.elf

What.

• Usercorn is an analysis and emulator framework, with a base similar to qemu-user.
• It can run arbitrary binaries on a different host kernel, unlike qemu-user.
• While recording full system state at every instruction.
• to a serializable compact format capable of rewind and re-execution.
• It's useful out of the box for debugging and dynamic analysis.
• With an arch-neutral powerful lua-based scripting language and debugger.
• It's also easy to extend and use to build your own tools.

Usercorn could be used to emulate 16-bit DOS, 32-bit and 64-bit ARM/MIPS/x86/SPARC binaries for Linux, Darwin, BSD, DECREE, and even operating systems like Redux.

Right now, x86_64 linux and DECREE are the best supported guests.

Why?

• Usercorn aims to be a framework to simplify emulating and deeply hooking a userspace environment for many target architectures and kernel ABIs.
• Debug stubborn binaries. I had a binary gdb refused to debug ("Program exited during startup."). No problem. Usercorn can single-step into the program for you.
• Debug foreign architecture and OS binaries. You don't need a MIPS box. You don't need qemu-user. You don't even need Linux.
• Write tools, like fuzzers, static analyzers, recompilers, memory and register analysis, overlay code coverage and machine state into IDA/Binary Ninja.
• Selectively call functions from within a binary. Usercorn will map a binary and emulate the kernel for you.
• Whatever you want. Open an issue if you have a cool debugging / reverse engineering idea I didn't think about - I may just implement it.

Caveats

• Your userspace might be incredibly confusing to the target binary.
• No API for memory mapped files yet (kinda, if mmap() currently gets a file descriptor argument it will manually copy the file into memory).
• I only have maybe 20% of the posix syscalls implemented, which is enough to run basic binaries. Busybox works great.

See Also (credit: XKCD)

usercorn's Issues

hangs on OS X after unicorn update

add syscall hook function per-arch/os

hooking insn_x86_syscall doesn't make any sense on arm/mips/etc.

could probably just make this an init function and remove the syscall hooks at root level.

deadlock detection

switch to ghostrace for syscall decoding

heuristic printf exploiter

identify stack variables pointing at the stack (or input/output buffer offset on the stack), automatically build a useful address chain for arbitrary memory control

ropwise disassembly

look for obvious rops in return (instructions that return to no call site)

currently segfaults on linux

Python: x86_64 darwin fails on exit

hello with printf: world
args (11):
01 @0x1000001 = "bins/x86_64.darwin.macho"
test: hi
strcmp: -11, 11, 01
file test 1: success

file test 2: success
success

Traceback (most recent call last):
File "_ctypes/callbacks.c", line 314, in 'calling callback function'
File "/usr/local/lib/python2.7/site-packages/unicorn/unicorn.py", line 249, in _hook_insn_syscall_cb
cb(self, data)
File "/Users/aegis/projects/usercorn/py/usercorn/cls.py", line 110, in hook_syscall
self.os.syscall(self.uc)
File "/Users/aegis/projects/usercorn/py/usercorn/arch/x64/darwin.py", line 26, in syscall
ret = syscalls.call(cls, SYSCALLS, num, lambda n: args[:n])
File "/Users/aegis/projects/usercorn/py/usercorn/syscalls.py", line 59, in call
return f(cls, *args) or 0
File "/Users/aegis/projects/usercorn/py/usercorn/syscalls.py", line 16, in write
return os.write(a1, cls.mem_read(a2, a3))
File "/usr/local/lib/python2.7/site-packages/unicorn/unicorn.py", line 189, in mem_read
data = ctypes.create_string_buffer(size)
File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/init.py", line 65, in create_string_buffer
buftype = c_char * init
ValueError: Array length must be >= 0, not -17958193

support dynamic linkers

• if type == EXEC, it's a normal executable
• if type == DYN, we mmap it somewhere
• also PT_INTERP gets vm_mmap()'d after elf_map runs and maps the whole elf
• so for exec'd linker, I need to start with DYN
• for INTERP ld-linux, I need to map the elf, then load and map the interpreter
• be careful to not map linker after the data segment so we can brk()

---

• support linux auxv
• support linux vdso
• support osx auxv equivalent (not sure 100% how the ABI plays out yet but dyld source is short)
• support osx commpage
• branch protect master when this is done and switch dev to an unstable branch

can do checksec.sh style stuff because I'm a loader

Add lib43 tests for all supported architectures

• Linux
  • x86-32
  • x86-64
  • arm
  • arm64
  • mips
• OS X
  • x86-32
  • x86-64
  • arm
  • arm64
  • ppc
• Windows ??
• DOS ??????????
• *BSD (use brandelf?)

"run until entry point"

invoke the interpreter to load dynamic libraries and stuff, then return or continue with hooks enabled at the real entry point

add tests that actually run a binary per arch

track valid addresses that have been loaded into registers

can also bound the addresses automatically based on increment vs store

array bound detection via accumulator too - if we're in a loop that's incrementing one or more memory addresses and affecting an accumulator in a proportional manner...

time traveling memory debugger

from a recorded trace, ability to select pages and slide across time - see register and memory state at specific time

annotate disassembler like ida with state of program at basic block entry, at register level, etc

"this instruction was hit 3000 times, with values X Y Z etc"

"this instruction touched these regions of memory [picture]" (with time slider to see when)

expand generic syscall backend

diff/overlay mem/reg/block/insn traces

sparc64?

add ASLR mapping emulation flag

consider how different operating systems would map the binary
also look at literally copying offsets from /proc/self/maps

update for unicorn binding changes

unicorn-engine/unicorn#149

sysctl support

Split out executable format backends?

Just need "is this your kind of file?", "symbolicate", and "return segments"?

MachO test fails

align diffed registers to their original location

this allows you to quickly glance down the row to compare values

switch to ghostrace for memio

add gdb stub?

how do I do breakpoints? dump an architecture specific trap at the target location?

interpreter symbols should be merged with program symbols for symbolication

Maybe have each loader return a list of symbols with Name string, Addr, Size uint64, Dynamic bool that get merged and handled by the Usercorn object.

mark status registers like sp/pc and exclude from reg diff

debug patching/tracing

run the program as normal, but trace specific memory access address(es), instructions

script a breakpoint and print buffers or registers at it, state diffing built in

symbolicate fails on ld-linux

Python: add cli args to enable execution tracing

Python: add strace

depends on #7

automatic stacktrace

every time we enter a new block, check the stack pointer
we can tell if we did ret or call based on this

replace python frontend with go shared object

MachO symbol lookup

generate syscall lists

I shouldn't need to maintain these by hand

• Linux/x86
• Linux/x86_64
• Linux/arm
• Linux/mips
• Linux/m68k
• Linux/sparc
• Darwin/x86
• Darwin/arm

Python: handle exceptions during syscall

Python: ARM segfault on map_segments()

If I don't read the data before passing to mem_write() (binascii.hexlify(data) works) I get a segfault.

split out syscall frontends

https://github.com/lunixbochs/usercorn/blob/master/usercorn.py#L325

Need generic syscall backend and frontends to decode for each arch/OS pair.

mips test currently hangs in generated code unless being single stepped

haven't found a reproduction case, happens on both linux and osx

add CGC frontend

Syscall ABI is per OS/arch, not just arch

symbolicate tagged memory region offsets

like esp + 0x4, ebp + 0x4, and [heap] + N, [libc.so.6] + N, etc

race condition / memory corruption with number of tuples?

Add smoke test for all architectures + travis config

• Smoke test
• Travis

SYSENTER on x86_32?

x86-32 linux test fails

calling convention codecs

does cffi have this?

two-column output mode

execution state on the left, registers and memory on the right

Python: does not run under pypy

this is important for performance