webassembly / wasp

WebAssembly module decoder in C++

wasp

Wasp is a C++ library designed to make it easy to work with WebAssembly modules. Unlike tools like wabt, it is designed to be used as a library.

It also includes the wasp tool, which has the following commands:

• wasp dump: Dump the contents of a WebAssembly module
• wasp callgraph: Generate a dot graph of the module's callgraph
• wasp cfg: Generate a dot graph of a function's control-flow graph
• wasp dfg: Generate a dot graph of a function's data-flow graph
• wasp validate: Validate a WebAssembly module
• wasp pattern: Find instruction sequence patterns
• wasp wat2wasm: Convert a Wasm text file to a Wasm binary file

Building using CMake (Linux and macOS)

You'll need CMake. You can then run CMake, the normal way:

$ mkdir build
$ cd build
$ cmake ..
$ cmake --build .

Building (Windows)

You'll need CMake. You'll also need Visual Studio (2019 or newer).

Note: Visual Studio 2017 and later come with CMake (and the Ninja build system) out of the box, and should be on your PATH if you open a Developer Command prompt. See https://aka.ms/cmake for more details.

You can run CMake from the command prompt, or use the CMake GUI tool. See Running CMake for more information.

When running from the commandline, create a new directory for the build artifacts, then run cmake from this directory:

> cd [build dir]
> cmake [wasp project root] -DCMAKE_BUILD_TYPE=[config] -DCMAKE_INSTALL_PREFIX=[install directory] -G [generator]

The [config] parameter should be a CMake build type, typically DEBUG or RELEASE.

The [generator] parameter should be the type of project you want to generate, for example "Visual Studio 16 2019". You can see the list of available generators by running cmake --help.

To build the project, you can use Visual Studio, or you can tell CMake to do it:

> cmake --build [wasp project root] --config [config] --target install

This will build and install to the installation directory you provided above.

So, for example, if you want to build the debug configuration on Visual Studio 2019:

> mkdir build
> cd build
> cmake .. -DCMAKE_BUILD_TYPE=DEBUG -DCMAKE_INSTALL_PREFIX=..\ -G "Visual Studio 16 2019"
> cmake --build . --config DEBUG --target install

wasp dump examples

Disassemble all functions in a module:

$ wasp dump -d mod.wasm

Display all sections in a module:

$ wasp dump -h mod.wasm

Display the contents of the "import" section:

$ wasp dump -j import -x mod.wasm

wasp callgraph examples

Write the callgraph as a DOT file to stdout.

$ wasp callgraph mod.wasm

Write the callgraph as a DOT file to file.dot.

$ wasp callgraph mod.wasm -o file.dot

You can use graphviz to convert the DOT file to an SVG:

$ dot -Tsvg file.dot -O

For example, the following wasm file:

(func $a call $b)
(func $b call $c call $d)
(func $c)
(func $d call $a call $b call $d)

Becomes this SVG:

wasp cfg examples

Write the CFG of function 0 as a DOT file to stdout.

$ wasp cfg -f 0 mod.wasm

Write the CFG of function foo as a DOT file to file.dot.

$ wasp cfg -f foo mod.wasm -o file.dot

For example, the following wasm file:

(func $fac (param i64) (result i64)
  (if (result i64) (i64.eq (local.get 0) (i64.const 0))
    (then (i64.const 1))
    (else
      (i64.mul (local.get 0) (call 0 (i64.sub (local.get 0) (i64.const 1)))))))

Becomes this SVG:

wasp dfg examples

Write the DFG of function 0 as a DOT file to stdout.

$ wasp dfg -f 0 mod.wasm

Write the DFG of function foo as a DOT file to file.dot.

$ wasp dfg -f foo mod.wasm -o file.dot

For example, the following wasm file:

(func $fac (param i64) (result i64)
  (if (result i64) (i64.eq (local.get 0) (i64.const 0))
    (then (i64.const 1))
    (else
      (i64.mul (local.get 0) (call 0 (i64.sub (local.get 0) (i64.const 1)))))))

Becomes this SVG:

wasp validate examples

Validate a module.

$ wasp validate mod.wasm

Validate multiple modules.

$ wasp validate mod1.wasm mod2.wasm mod3.wasm

wasp pattern examples

Print the 10 most common instruction sequences.

$ wasp pattern mod.wasm -d 10

This produces results similar to those shown below. The columns are as follows:

1. How often the instruction sequence occurred
2. The instruction sequence length
3. The instruction sequence
4. A percentage, calculated as 100 * count * sequence length / total instructions.

71333: [2] [i32.const 0 global.set 10] 4.17%
37382: [2] [end end] 2.19%
34937: [2] [i32.const 1 i32.and] 2.04%
25099: [2] [block [] block []] 1.47%
21440: [2] [i32.and if []] 1.25%
20154: [2] [i32.eqz if []] 1.18%
19883: [3] [i32.const 1 i32.and if []] 1.75%
18643: [2] [return end] 1.09%
15857: [3] [block [] block [] block []] 1.39%
15314: [2] [end local.get 0] 0.90%
total instructions: 3417737

wasp wat2wasm examples

Convert test.wat to test.wasm.

$ wasp wat2wasm test.wat

Convert test.wat to something.wasm.

$ wasp wat2wasm test.wat -o something.wasm

Convert test.wat to test.wasm, but skip validation.

$ wasp wat2wasm test.wat --no-validate

Convert test.wat to test.wasm, and enable the SIMD feature.

$ wasp wat2wasm test.wat --enable-simd

wasp's Issues

[gc] Add support for `rtt.canon` etc. in global initializers

For example:

(module
  (type $Super (struct (field $x (mut i32))))

  (global $Rtt (rtt 1 $Super) (rtt.canon $Super))
)

This also will require extending global initializers to multiple instructions, e.g.:

(rtt.sub ... (rtt.canon ...))

ref.func incorrectly gives "Undeclared function reference" if there is no reference to the function in a global (wat2wasm)

The following code

(module
;;(type $t (func))
;;(global (ref $t) (ref.func $f))
(func $f (block
  (ref.func $f)
  (drop)
))
)

gives the following error:

/tmp/test.wat:5:4: Undeclared function reference 0
  (ref.func $f)

Assigning the reference to a global (uncommenting the commented code) allows the reference to the function to be taken in the code below with no error.

There is a comment in validate.cc:205 which seems suspect

        // ref.func indexes are implicitly declared by referencing them in a
        // constant expression.

Use more warnings (at least for CI)

See #35 (comment).

In particular, -Wsuggest-override, but perhaps there are others.

Add tests for tools

The callgraph, cfg, dfg, dump tools don't have any tests. This shouldn't be too much work to do. If we refactor the Tool class to use an abstract output (instead of writing to file/stdout) then it should work.

LLVM version dependency in parallel-hashmap?

I'm trying to get wasp to build, so I can report my numbers for WebAssembly/design#1277. So far, I've done the following:

Environment

OS: macOS Mojave 10.14.6

Steps

1. git clone [email protected]:binji/wasp.git binji/wasp
2. cd binji/wasp
3. git submodule update --init --recursive
4. make

Error

In file included from /Users/luke.imhoff/github/binji/wasp/third_party/parallel-hashmap/parallel_hashmap/phmap.h:49:
/Users/luke.imhoff/github/binji/wasp/third_party/parallel-hashmap/parallel_hashmap/phmap_utils.h:139:16: error: implicit instantiation of undefined template
      'std::__1::hash

Full Log

mkdir -p out/clang/Debug/
cd out/clang/Debug/ && cmake -G "Unix Makefiles" /Users/luke.imhoff/github/binji/wasp/ -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Debug 
-- The C compiler identification is Clang 7.0.1
-- The CXX compiler identification is Clang 7.0.1
-- Check for working C compiler: /usr/local/opt/llvm/bin/clang
-- Check for working C compiler: /usr/local/opt/llvm/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/local/opt/llvm/bin/clang++
-- Check for working CXX compiler: /usr/local/opt/llvm/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python (found version "2.7.10") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/luke.imhoff/github/binji/wasp/out/clang/Debug
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory -C out/clang/Debug/ all
Scanning dependencies of target wasplib
[  1%] Building CXX object CMakeFiles/wasplib.dir/src/base/features.cc.o
[  2%] Building CXX object CMakeFiles/wasplib.dir/src/base/file.cc.o
[  3%] Building CXX object CMakeFiles/wasplib.dir/src/base/span.cc.o
[  4%] Building CXX object CMakeFiles/wasplib.dir/src/base/str_to_u32.cc.o
[  5%] Building CXX object CMakeFiles/wasplib.dir/src/base/v128.cc.o
[  6%] Building CXX object CMakeFiles/wasplib.dir/src/binary/br_on_exn_immediate.cc.o
[  7%] Building CXX object CMakeFiles/wasplib.dir/src/binary/br_table_immediate.cc.o
[  8%] Building CXX object CMakeFiles/wasplib.dir/src/binary/call_indirect_immediate.cc.o
[  9%] Building CXX object CMakeFiles/wasplib.dir/src/binary/code.cc.o
[ 10%] Building CXX object CMakeFiles/wasplib.dir/src/binary/comdat.cc.o

[ 11%] Building CXX object CMakeFiles/wasplib.dir/src/binary/comdat_symbol.cc.o
In file included from /Users/luke.imhoff/github/binji/wasp/src/binary/comdat_symbol.cc:20:
In file included from /Users/luke.imhoff/github/binji/wasp/src/base/std_hash_macros.h:20:
In file included from /Users/luke.imhoff/github/binji/wasp/include/wasp/base/hash.h:22:
In file included from /Users/luke.imhoff/github/binji/wasp/third_party/parallel-hashmap/parallel_hashmap/phmap.h:49:
/Users/luke.imhoff/github/binji/wasp/third_party/parallel-hashmap/parallel_hashmap/phmap_utils.h:139:16: error: implicit instantiation of undefined template
      'std::__1::hash<wasp::binary::ComdatSymbolKind>'
        return std::hash<T>()(val);
               ^
/Users/luke.imhoff/github/binji/wasp/third_party/parallel-hashmap/parallel_hashmap/phmap_utils.h:144:16: note: in instantiation of function template specialization
      'phmap::Hash<wasp::binary::ComdatSymbolKind>::_hash<wasp::binary::ComdatSymbolKind, 0>' requested here
        return _hash<T>(val);
               ^
/Users/luke.imhoff/github/binji/wasp/third_party/parallel-hashmap/parallel_hashmap/phmap_utils.h:302:48: note: in instantiation of member function
      'phmap::Hash<wasp::binary::ComdatSymbolKind>::operator()' requested here
                                         seed, phmap::Hash<T>()(v)),  vs...);
                                               ^
/Users/luke.imhoff/github/binji/wasp/src/binary/comdat_symbol.cc:30:1: note: in instantiation of function template specialization 'phmap::HashStateBase<unsigned
      long>::combine<wasp::binary::ComdatSymbolKind, unsigned int>' requested here
WASP_STD_HASH_2(::wasp::binary::ComdatSymbol, kind, index)
^
/Users/luke.imhoff/github/binji/wasp/src/base/std_hash_macros.h:37:31: note: expanded from macro 'WASP_STD_HASH_2'
    return ::wasp::HashState::combine(0, v.f1, v.f2);  \
                              ^
/usr/local/Cellar/llvm/7.0.1/include/c++/v1/type_traits:412:50: note: template is declared here
template <class _Tp> struct _LIBCPP_TEMPLATE_VIS hash;
                                                 ^
1 error generated.
make[3]: *** [CMakeFiles/wasplib.dir/src/binary/comdat_symbol.cc.o] Error 1
make[2]: *** [CMakeFiles/wasplib.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [clang-debug] Error 2

Any tips for how to get parallel-hashmap to compile?

Support for GC Milestone 3

It look like that WASP implements currently milestone 2. This partly incompatible with milestone 3.

https://docs.google.com/document/d/1DklC3qVuOdLHSXB5UXghM_syCh-4cMinQ50ICiXnK3Q/edit#heading=h.9dwoku9340md

How to install WASP

Hello,
I'm new to webassembly and wasp. As part of my project im trying to get to use wasp, currently I do have wabt installed, I runned test through it. Now I want to create call graph, data flow graph using wasp, put I cant install it and I dont know what should do.
Please help

Add validate tool

Most of the validation logic is finished, it just needs to be wired up to a new tool.

Error on roundtripping a testcase with "let"

(module
  (type $vector (array (field (mut f64))))
  (func $main
    (local $x i32)
    (local $y i32)
    (drop (local.get $x)) ;; 0 is the index appearing in the binary
    ;; first let
    (array.new_with_rtt $vector
      (f64.const 3.14159)
      (i32.const 1)
      (rtt.canon $vector)
    )
    (let (local $v (ref $vector))
      (drop (local.get $v)) ;; 0
      (drop (local.get $x)) ;; 1
      ;; another one, nested
      (array.new_with_rtt $vector
        (f64.const 1234)
        (i32.const 2)
        (rtt.canon $vector)
      )
      (let (local $w (ref $vector))
        (drop (local.get $v)) ;; 1
        (drop (local.get $w)) ;; 0
        (drop (local.get $x)) ;; 2
      )
    )
    ;; another one, later
    (array.new_with_rtt $vector
      (f64.const 2.1828)
      (i32.const 3)
      (rtt.canon $vector)
    )
    (let (local $v (ref $vector))
      (drop (local.get $v)) ;; 0
      (drop (local.get $x)) ;; 1
    )
    (drop (local.get $x)) ;; 0
  )
)

wat2wasm works, but running wasm2wat on the output hits

<unknown>:0000005d: Unexpected end instruction after 'end'
    2002 1a0b 0b44 9fab add8 5f76 01
              ^^                    

(testcase was written for WebAssembly/binaryen#3485)

Check section ordering/duplication in binary reader

Right now the LazyModule will read any section in any order. This is useful for tools like dump, but in general we'll want to make it easy to check the section ordering. Maybe the best way is to duplicate the code in LazySequence and add some additional checks there.

Type casting does not work with complex sample

The follow simple sample:

(module
  (type $java/lang/Object (struct
    (field $java/lang/Object..vtable (mut i32))
    (field $java/lang/Object..hashcode (mut i32))
  ))
  (type $java/lang/Class (struct
    (field $java/lang/Class..vtable (mut i32))
    (field $java/lang/Class..hashcode (mut i32))
    (field $java/lang/Class.vtable (mut i32))
  ))
  (func $java/lang/Object.<init>(param $this (ref null $java/lang/Object))
    return
  )
  (func $java/lang/Class.<init>(param $this (ref null $java/lang/Class))(param $vtable i32)
    local.get $this
    call $java/lang/Object.<init>
    local.get $this
    local.get $vtable
    struct.set $java/lang/Class 2 ;; $vtable
    return
  )
)

can be converted without any errors with the command line: wasp wat2wasm --enable-gc --enable-sign-extension test.wat

With a more complex sample test.zip I received the follow error:

test.wat:2817:5: Expected stack to contain [(ref null 86)], got [(ref null 89)]
    call $java/lang/Object.<init>
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It look like that it can't compare the different types correctly if there are to many types. If there is an error in the sample which I can't see then a better error message would be desirable.

Preparing for integration into wabt

See WebAssembly/wabt#1543.

There are still some missing features from wasp that we'll want before we can use it to fully replace the equivalent wabt functionality:

• binary to text conversion
• memory64 support
• missing linking section features
  • memory64 relocations
  • check for any other differences (linking section in wasp was implemented a year ago or so, make sure nothing else needs updating)
• generating names in text format
• folded text output
• documentation
  • top-level directory structure
  • document file headers
  • document all structs/classes

Update the exception handling support to the current proposal

The exception handling has changed. The follow simple example should work but produce errors with the current version. V8 already use the new exception handling.

(module
  (event (param externref))
  (func $test
    try
	catch 0
	  drop
	end
  )
)

rtt.sub has 1 heaptype immediate

In the GC prototype 2, rtt.sub has 2 heaptype immediates, while wasp seems to emit 1.

Noticed while comparing the behavior of v8, wasp, and binaryen.

cc @jakobkummerow

Canont install wasp, there no executable file in `bin`

When I finish the instructions, there is nothing in ~/wasp/build/bin/

mkdir build
cd build
cmake ..
cmake --build .

Invalid event index 0, must be less than 0

I receive the follow error with the wasp command line tool:

wasp validate --enable-gc --enable-exceptions test.wasm
[FAIL] test.wasm
<unknown>:00000f0e: Invalid event index 0, must be less than 0
    0210 5408 000b 2001 4100 4804 40
              ^^

Decompiled look the code position:

000f0d: 08 00                      |   throw 0

My meanings is that the index for an event/exception must be >= 0 that I interpret this error message that the list of exception definitions is zero. But the event section look like:

Contents of section event:
00008fb: 0100 8201

This will also correct decompiled as:

(event (type 130))

I think it is a bug in of the wasp validate message or I oversee some important things.

test.zip

When reporting errors, use names instead of indexes if they are provided.

wat2wasm errors are cumbersome to debug since they only provide index numbers instead of their symbolic names.

C++ Runtime error

With the follow command wasp validate test.wasm I receive the follow error:

testwasmfile.zip

How to build on Windows?

I try to build it but with no luck.

C:\git\wasp>cmake C:\git\wasp
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.19041.
-- The C compiler identification is MSVC 19.22.27905.0
-- The CXX compiler identification is MSVC 19.22.27905.0
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx86/x86/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx86/x86/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx86/x86/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.22.27905/bin/Hostx86/x86/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:267 (add_subdirectory):
  add_subdirectory given source "third_party/gtest" which is not an existing
  directory.


-- Configuring incomplete, errors occurred!
See also "C:/git/wasp/CMakeFiles/CMakeOutput.log".

CMakeOutput.log

Text syntax for array type definition

The follow code for an array definition:

(module
  (type $bytes (array (mut i8)))
)

produce the follow error:

test.wat:2:23: Expected '(' Field, got Lpar Mut
  (type $bytes (array (mut i8)))
                      ^
test.wat:2:23: Expected Rpar, got Lpar
  (type $bytes (array (mut i8)))
                      ^
test.wat:2:23: Expected Eof, got Lpar
  (type $bytes (array (mut i8)))
                      ^

With a field expression which is unexpected for arrays it will work:

(module
  (type $bytes (array (field (mut i8))))
)

see also: WebAssembly/gc#172

PS: The error message also suggest an uppercase Field but it must be a lowercase field.

wasp encodes element sections differently than binaryen [typed function references]

The example func_ref_o.wat:

(module
  (func (export "main") (result i32)
    (call_ref (i32.const 10)
              (ref.func $foo)
    )
  )
  (func $foo (param $x i32) (result i32)
    (i32.add (local.get $x)
             (i32.const 19))
  )
  (elem declare funcref (ref.func $foo))
)

wasp wat2wasm --enable-function-references:

Contents of section element:
0000025: 0107 7001 d201 0b                        ..p....

binaryen wasm-as --enable-reference-types:

Contents of section element:
0000025: 0103 0001 01                             .....

wasp encodes element section as vec(expr) and binaryen encodes it as vec(func_idx), according to the https://github.com/WebAssembly/function-references/blob/master/proposals/function-references/Overview.md there is no any evidence that vec(expr) should be used.

Is it a bug?

Section out of order: global cannot occur after event

I receive the message: Section out of order: global cannot occur after event

• But this is the right section order if I look into the exception proposal: https://github.com/WebAssembly/exception-handling/blob/master/proposals/exception-handling/Exceptions.md. I have use the command wasp dump -h test.wasm

• Also ist the section today names Exception and not more Event

• I receive also: Unknown section id: 13
  with the command wasp validate --enable-gc test.wasm
  13 is the valid ID for the exception section.