GithubHelp home page GithubHelp logo

google / bindiff Goto Github PK

View Code? Open in Web Editor NEW
2.1K 28.0 121.0 316.17 MB

Quickly find differences and similarities in disassembled code

Home Page: https://zynamics.com/bindiff.html

License: Apache License 2.0

CMake 0.27% C++ 27.85% Java 71.32% Shell 0.40% Makefile 0.02% Rich Text Format 0.02% Batchfile 0.10% Dockerfile 0.02%
bindiff binexport diffing program-analysis program-differencing reverse-engineering c-plus-plus ida-plugin ida-pro java

bindiff's Introduction

BinDiff Logo

Copyright 2011-2024 Google LLC.

BinDiff

This repository contains the BinDiff source code. BinDiff is an open-source comparison tool for binary files to quickly find differences and similarities in disassembled code.

Table of Contents

About BinDiff

BinDiff is an open-source comparison tool for binary files, that assists vulnerability researchers and engineers to quickly find differences and similarities in disassembled code.

With BinDiff, researchers can identify and isolate fixes for vulnerabilities in vendor-supplied patches. It can also be used to port symbols and comments between disassemblies of multiple versions of the same binary. This makes tracking changes over time easier and allows organizations to retain analysis results and enables knowledge transfer among binary analysts.

Use Cases

  • Compare binary files for x86, MIPS, ARM, PowerPC, and other architectures supported by popular disassemblers.
  • Identify identical and similar functions in different binaries
  • Port function names, comments and local names from one disassembly to the other
  • Detect and highlight changes between two variants of the same function

Quickstart

If you want to just get started using BinDiff, download prebuilt installation packages from the releases page.

Note: BinDiff relies on a separate disassembler. Out of the box, it ships with support for IDA Pro, Binary Ninja and Ghidra. The disassemblers page lists the supported configurations.

Documentation

A subset of the existing manual is available in the docs/ directory.

Codemap

BinDiff contains the following components:

  • cmake - CMake build files declaring external dependencies
  • fixtures - A collection of test files to exercise the BinDiff core engine
  • ida - Integration with the IDA Pro disassembler
  • java - Java source code. This contains the the BinDiff visual diff user interface and its corresponding utility library.
  • match - Matching algorithms for the BinDiff core engine
  • packaging - Package sources for the installation packages
  • tools - Helper executables that are shipped with the product

Building from Source

The instruction below should be enough to build both the native code and the Java based components.

More detailed build instructions will be added at a later date. This includes ready-made Dockerfiles and scripts for building the installation packages.

Native code

BinDiff uses CMake to generate its build files for those components that consist of native C++ code.

The following build dependencies are required:

  • BinExport 12, the companion plugin to BinDiff that also contains a lot of shared code
  • Boost 1.83.0 or higher (a partial copy of 1.83.0 ships with BinExport and will be used automatically)
  • CMake 3.14 or higher
  • Ninja for speedy builds
  • GCC 9 or a recent version of Clang on Linux/macOS. On Windows, use the Visual Studio 2019 compiler and the Windows SDK for Windows 10.
  • Git 1.8 or higher
  • Dependencies that will be downloaded:
    • Abseil, GoogleTest, Protocol Buffers (3.14), and SQLite3
    • Binary Ninja SDK

The following build dependencies are optional:

  • IDA Pro only: IDA SDK 8.2 or higher (unpack into deps/idasdk)

The general build steps are the same on Windows, Linux and macOS. The following shows the commands for Linux.

Download dependencies that won't be downloaded automatically:

mkdir -p build/out
git clone https://github.com/google/binexport build/binexport
unzip -q <path/to/idasdk_pro80.zip> -d build/idasdk

Next, configure the build directory and generate build files:

cmake -S . -B build/out -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=build/out \
  -DBINDIFF_BINEXPORT_DIR=build/binexport \
  "-DIdaSdk_ROOT_DIR=${PWD}build/idasdk"

Finally, invoke the actual build. Binaries will be placed in build/out/bindiff-prefix:

cmake --build build/out --config Release
(cd build/out; ctest --build-config Release --output-on-failure)
cmake --install build/out --config Release

Building without IDA

To build without IDA, simply change the above configuration step to

cmake -S . -B build/out -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=build/out \
  -DBINDIFF_BINEXPORT_DIR=build/binexport \
  -DBINEXPORT_ENABLE_IDAPRO=OFF

Java GUI and yFiles

Building the Java based GUI requires the commercial third-party graph visualisation library yFiles for graph display and layout. This library is immensely powerful, and not easily replaceable.

To build, BinDiff uses Gradle 6.x and Java 11 LTS. Refer to its installation guide for instructions on how to install.

Assuming you are a yFiles license holder, set the YFILES_DIR environment variable to a directory containing the yFiles y.jar and ysvg.jar.

Note: BinDiff still uses the older 2.x branch of yFiles.

Then invoke Gradle to download external dependencies and build:

Windows:

set YFILES_DIR=<path\to\yfiles_2.17>
cd java
gradle shadowJar

Linux or macOS:

export YFILES_DIR=<path/to/yfiles_2.17>
cd java
gradle shadowJar

Afterwards the directory ui/build/libs in the java sub-directory should contain the self-contained bindiff-ui-all.jar artifact, which can be run using the standard java -jar command.

Further reading / Similar tools

The original papers outlining the general ideas behind BinDiff:

  • Thomas Dullien and Rolf Rolles. Graph-Based Comparison of Executable Objects. bindiffsstic05-1.pdf. SSTIC ’05, Symposium sur la Sécurité des Technologies de l’Information et des Communications. 2005.
  • Halvar Flake. Structural Comparison of Executable Objects. dimva_paper2.pdf. pp 161-173. Detection of Intrusions and Malware & Vulnerability Assessment. 2004.3-88579-375-X.

Other tools in the same problem space:

  • Diaphora, an advanced program diffing tool implementing many of the same ideas.
  • TurboDiff, a now-defunct program diffing plugin for IDA Pro.

Projects using BinDiff:

  • VxSig, a tool to automatically generate AV byte signatures from sets of similar binaries.

License

BinDiff is licensed under the terms of the Apache license. See LICENSE for more information.

Getting Involved

If you want to contribute, please read CONTRIBUTING.md before sending pull requests. You can also report bugs or file feature requests.

bindiff's People

Contributors

cblichmann avatar cgdecker avatar chcunningham01 avatar ckennelly avatar eltociear avatar gamozolabs avatar hsudhof avatar igotti-google avatar jaegeral avatar latortuga71 avatar lkwiatek avatar manouchehri avatar mkruskal-google avatar patriq avatar pistonminer avatar pratikpc avatar rmngoog avatar thracky avatar ubfx avatar you0708 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bindiff's Issues

Ignore JMP and treat JX==JNX when diffing basic blocks

Steps to reproduce the problem:

  1. Diff a program
  2. Open a function with some differences
  3. See some basic blocks which are marked as different only because of the jump at the end

What is the expected behavior?
See the following screenshot:
0j2Y7jD
In such cases, it would be helpful for the blocks to be marked as equal.

What went wrong?
They are marked as different.

What version of the product are you using? On what operating system?
BinDiff 6, Win10.

Ported from b/175509898

Export Functionality for Matched Functions in BinDiff

Is your feature request related to a problem? Please describe.

Currently, when using BinDiff for binary file comparison, we can see the matched functions, but there's no convenient way to export this information for further analysis or processing. This leads to inefficiency when we need to perform additional operations on a large number of matched functions, especially when dealing with large projects or when we need to integrate the results with other tools.

Describe the solution you'd like

I would like BinDiff to add an export functionality that allows users to export all matched functions to a file. Specifically:

  1. Add an "Export Matched Functions" option in the BinDiff interface, which could be a button or a menu item.
  2. When this option is clicked, allow users to choose the export format (such as CSV, JSON, or XML) and the save location.
  3. The exported file should include key information for each matched function, such as function name, address, similarity score, etc.
  4. Ideally, users should be able to customize which fields they want to export.

This feature would greatly improve analysis efficiency and make it easier to integrate BinDiff results with other tools.

Describe alternatives you've considered

  1. Manual copy and paste: Currently, it's possible to "export" data by manually copying information from the BinDiff interface and pasting it into a text editor, but this process is time-consuming and prone to errors.

  2. Using screen capture tools: Screen capture tools could be used to capture BinDiff results, but this method is not precise and difficult to handle large amounts of data.

  3. Developing third-party scripts: We've considered developing a script to read BinDiff output files and extract the required information, but this requires additional development work and may become obsolete with BinDiff version updates.

Additional context

This feature would be particularly helpful in the following scenarios:

  • Version comparison of large software projects that require analysis of numerous matched functions.
  • Security research, where BinDiff results need to be integrated with other vulnerability analysis tools.
  • Automated workflows, where BinDiff results serve as input for other analysis steps.

If an API or command-line interface could be provided for this export functionality, it would be even more beneficial for integrating BinDiff into automated workflows.

BinDiff should use Unicode APIs on Windows

Steps to reproduce the problem:
1.saved two ida database.
2.Open one of the two database to diff the other database
3.alert message as "Error while diffing: failed opening database: 'unable to open database file', filename: 'C:\Users\���\AppData\Local\Temp\BinDiff\temporary.database' "

What is the expected behavior?
I wish that the Bindiff can support Chinese.

What went wrong?
I can't diff the two database.

What version of the product are you using? On what operating system?
Bindiff 7 beta. On Windows.
Any other comments?

186791621__33198687__191454

Ported from b/186791621

Reading raw primary call graph - forever

after load workspace
i try load my diff by pressing twice by name

i see message box progress dialog

Loading Diff 'my_vs_my1'
Reading raw primary call graph...'

and thats all

when it start, java.exe consume over 99% CPU and around 1Gb memory
after few seconds its rollback to normal

but progress loading infinet
i see only seconds tick "9999 seconds" etc

iam try update jre but it doest help

any idea how to debug issue ?

Need aarch64 release, for both macOS and Linux

Is your feature request related to a problem? Please describe.
There is no release for aarch64 architecture

Describe the solution you'd like
Please release a package which is compiled for aarch64 architecture specially for linux (debian, arch). am64 releases includes binary plugins for IDA Pro, so they are also in amd64 release.

Stuck in "Exporting primary IDB", waited for a long time with no progress.

device Info

  • Mac Mini - Apple M1 - Sonoma 14.0
  • BinDiff 8

screenshot

image
image

Mark

These are two small test programs I wrote, and I wanted to compare the differences using BinDiff. However, it has been stuck in the 'Exporting primary IDB' state for the past two hours, and I haven't received any results. I would appreciate your guidance on this matter.

IDA plugin - What if we only want to import function/symbol names?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Not everyone needs both the function/symbol names AND comments from the database. I unfortunately didn't see an easy way to configure to basically ignore importing comments.

Describe the solution you'd like
A clear and concise description of what you want to happen.

When selecting the Import Names and Comments... button (maybe you can rename the button to be something more generic like Import Data) then having a popup dialog box where the user can select which items they want imported into the database. Also, having to check the same boxes on every opening of the dialog would be annoying, so having these selected items be persisted might be nice?

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Well I guess I could make a duplicate database and clear out the comments only and import it from there, but where would be the fun in that?

Additional context
Add any other context or screenshots about the feature request here.

The BinDiff core engine should have a test suite with reasonable coverage

BinDiff traditionally was only tested end-to-end running it on sets of BinExport files. While there is a large body of golden tests with expected results, any functional change to the core invalidates the existing test set. This makes implementation of new matching features or fixing long-standing issues unnecessarily hard.

Implement a test suite the has reasonable code coverage for the BinDiff core engine.

Internal bug: b/147221242

Can subviews be reopened?

After running a diff, if I accidentally close the Matched Functions subview is there a way to reopen it?

subview

I don't see any menu or plugin options for reopening the subviews. I also checked the documentation but I don't see anything there, either.

The workaround I was able to come up with was to save and load the results but I wish there was a simpler way.

save-load

`bindiff --ui` should bring existing UI to foreground

Steps to reproduce the problem:

  1. Run bindiff --ui
  2. Run bindiff --ui again

What is the expected behavior?

One instance of the UI should open and the second invocation should bring it into foreground.

What went wrong?

The second BinDiff UI exits, because it cannot listen on its socket server port.

What version of the product are you using? On what operating system?

BinDiff 7, any OS

Any other comments?

Likely implementation not in core, but in UI itself.

Rebuild the UI to use web technologies for easier embedding

Is your feature request related to a problem? Please describe.

  • The Java UI is dated and requires a proprietary graph visualization libray
  • BinDiff native plugins for IDA Pro and binary Ninja cannot embedd a high-quality graph view
  • The current workflow with the standalone UI is cumbersome and few people actually use "BinDiff Workspaces".

Describe the solution you'd like

  • Implement a UI using Typescript and web technologies. D3.js with the DAG layouter should be able to support the flow graph use cases and render to Canvas.
  • Have the bindiff executable launch a server for this and either embed a web view in the disassemblers or show the browser.

Describe alternatives you've considered

  • Gephi
  • Extracting the graphing library from Ghidra
  • Implementing a new desktop UI in Qt

Additional context
This should eventually become a "GitHub Project".

Linker issue with LTO symbols when using XCode 15.0.1 (or later)

Describe the bug
Using a Mac with XCode 15.0, configuring the CMake build and running cmake --build . --config Release (internally, this uses a the Makefile generator) gives this error:

...
[ 93%] Built target bindiff
0  0x109dabf43  __assert_rtn + 64
1  0x109d53a20  mach_o::Symbol ld::Layout::symbolForAtom<ld::LinkedAtomPlacement const>(ld::Atom const*, CString, ld::LinkedAtomPlacement const&, ld::DylibMapping const*, unsigned long long) const + 2384
2  0x109d4d44b  void dispatchForEach<ld::AtomAndName const, void ld::buildSymbolTable<ld::LinkedAtomPlacement>(ld::SymbolTableLayout&, ld::Layout const&, ld::Options const&, ld::AtomSymbolPartition const&, ld::LinkedAtomPlacement const&, ld::DylibMapping const*, unsigned long long, bool, bool)::'lambda1'(unsigned long, ld::AtomAndName const&)>(std::__1::span<ld::LinkedAtomPlacement, 18446744073709551615ul>, unsigned long, void ld::buildSymbolTable<ld::LinkedAtomPlacement>(ld::SymbolTableLayout&, ld::Layout const&, ld::Options const&, ld::AtomSymbolPartition const&, ld::LinkedAtomPlacement const&, ld::DylibMapping const*, unsigned long long, bool, bool)::'lambda1'(unsigned long, ld::AtomAndName const&))::'lambda'(unsigned long)::operator()(unsigned long) const + 187
3  0x109d5895b  ld::LayoutExecutable::writeToFile(char const*) + 9099
4  0x109cfaf9f  main + 12831
ld: Assertion failed: (0 && "lto symbol should not be in layout"), function symbolForAtom, file Layout.cpp, line 1381.
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [call_graph_test] Error 1
make[1]: *** [CMakeFiles/call_graph_test.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
ld: warning: ignoring duplicate libraries: '../_deps/absl-build/absl/time/libabsl_time.a'
ld: warning: ignoring duplicate libraries: '../_deps/absl-build/absl/time/libabsl_time.a'
ld: warning: ignoring duplicate libraries: '../_deps/absl-build/absl/time/libabsl_time.a'
[ 93%] Built target change_classifier_test
ld: warning: ignoring duplicate libraries: '../_deps/absl-build/absl/time/libabsl_time.a'
[ 94%] Built target bindiff8_ida.dylib
[ 97%] Built target bindiff8_ida64.dylib
make: *** [all] Error 2

To Reproduce
Steps to reproduce the behavior:

  1. See above, configure CMake and run cmake --build . --config Release

Expected behavior
Release binaries of BinDiff being built.

Screenshots
N/A

Environment (please complete the following information):

  • OS and version: macOS Sonoma, XCode 15.0
  • Locale: US English
  • Disassembler and version: N/A

Additional context
As a workaround, the "classic" linker ld64 can be used instead of the new ld_prime one. See the XCode 15.0 Release Notes.

fit "zoom to basicblock" to reading symbols

now "zoom to basicblock" fit to block inside the diff window

its not comfortable, there are need to many press "zoom in" to allow reading symbols

more usefull will be fit blocks to reading symbols by "zoom to basicblock"

look at the pics

this is zoom fit for now

bindiff_zoom_fit

this is zoom fit which i need

bindiff_zoom_fit_ok

UI: After adding a "New Diff", the diff should be automatically opened

Steps to reproduce the problem:

Open BinDiff, use Diffs|New Diff to create a new diff
What is the expected behavior?

  1. The newly added diff should automatically be opened/loaded by the UI.

What went wrong?

One has to double click the new diff in the tree view for it to be loaded.

What version of the product are you using? On what operating system?

BinDiff 7, any OS

Any other comments?

N/A

Can't Diff and Port same named executables at the same time

Do to the way Binexport and/or Bindiff names it's temp files, you can not successfully do more than one diff and/or port at the same time.

The problem is because the tools use the same temp folder and the temp DBs that are created have the same names since they use the the original base name of the executable. Plus a common/global DB named "temporary.database" is already in use.

Example session for the IDB name problem:
If you made an IDB of an executable named "foobar.exe" and saved it "foobar_a.exe.i64" and another of the same named exe (from an update or something) "foobar_b.exe.i64" the tools will name them all "foobar.BinExport" since the tools name these internally based on the base name.
Normally not a problem for a single IDB operation since the DB files are separated path structure "BinDiff\primary" and "BinDiff\secondary". But if you open one more additional sessions (different "ida64.exe" processes) with "foobar_aa.exe.i64" and "foobar_bb.exe.i64" now the conflict exists because these names are taken plus "temporary.database" is already used.

Steps to reproduce the problem:

  1. The normal modus operandi, BinDiff an older executable with a new then port symbols and comments using the port feature. But attempting to do more than one at the same time.

What is the expected behavior?
Expected to be able to Diff and port two different IDA IDBs at the same time on the same machine.

What went wrong?
At least the port feature will fail when it is started, and probably results in diff corruption.

What version of the product are you using? On what operating system?
"BinDiff 7 (latest version, updated)", IDA 7.6 and IDA 7.5 on Windows 10 21h1

Any other comments?
As a solution, consider adding a top level root folder called "Session438" or something where the "438" number is the PID of the host "ida64.exe". An easy way to serialize the the folder structure with a UUID.

Ported from b/200299139

Enable building without the IDA SDK

Is your feature request related to a problem? Please describe.
I have no access to the IDA SDK (downloading it appears to require an IDA Pro license/login), but I have no need for the IDA Pro plugin anyways, so I'd like to be able to build the bindiff executable without having to obtain the IDA SDK.

Describe the solution you'd like
BinExport can be built without the IDA SDK by setting -DBINEXPORT_ENABLE_IDAPRO=NO, so it would be nice to have a similar setting for BinDiff (e.g., BINDIFF_ENABLE_IDAPRO). It would be even more convenient if the build system could automatically detect the presence or absence of the IDA SDK, so it wouldn't be necessary to set a flag in the first place.

Describe alternatives you've considered
I can manually patch the CMakeLists.txt files, but it's not a particularly user-friendly solution.

IDA Pro: `Shift-D` hotkey interferes with search

Describe the bug
In IDA Pro, pressing the Shift-D hotkey while focussed in one of IDA's list windows (like the Functions list) interferes with the search functionality.

To Reproduce
Steps to reproduce the behavior:

  1. Click on any function in the 'Functions' list
  2. Start typing
  3. Type an uppercase 'D'

Expected behavior
IDA continues the find-as-you-type action

Screenshots
N/A

Environment (please complete the following information):

  • OS and version: any
  • Locale: any
  • Disassembler and version: IDA Pro 8.3, but all versions are affected

Additional context
A global hotkey for BinDiff would be useful.

Variable names are not being imported anymore

Steps to reproduce the problem:

  1. Diff against a database that has local or global variable names
  2. Import the comments & symbol
  3. Check if the variables have been ported over

What is the expected behavior?

The variables in the matched functions should be ported over.

What went wrong?

The variables are not ported over.

What version of the product are you using? On what operating system?

  • Windows 11 64bit
  • Ida Pro 7.6 SP1
  • BinDiff 7

Ported from b/206972653

Bindiff unable to process statically linked binaries exported with BinExport from Ghidra (flow graph already attached error)

Describe the bug
Bindiff is unable to process statically linked binaries exported by Ghidra with BinExport.

I have not tried BinExport with IDA.

To Reproduce
Steps to reproduce the behavior:
1.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
A binary like this:
openssl: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=6143bc37788cd04527de063e16c35627b1610ea2, for GNU/Linux 3.2.0, not stripped

Will result in an error like this:
terminate called after throwing an instance of 'std::runtime_error'
E0215 14:29:31.721843 1785634 flow_graph.cc:85] Could not find basic block: 008D6BF0
E0215 14:29:31.721862 1785638 flow_graph.cc:85] Could not find basic block: 005A8E40
E0215 14:29:31.721827 1785635 flow_graph.cc:85] Could not find basic block: 0068A377
E0215 14:29:31.721854 1785637 flow_graph.cc:85] Could not find basic block: 00654090
E0215 14:29:31.721843 1785639 flow_graph.cc:85] Could not find basic block: 008E6230
what(): AttachFlowGraph: flow graph already attached 00401000

Environment (please complete the following information):

  • Ubuntu 23.10
  • Locale: US English
  • Disassembler and version: Ghidra 11.0 with BinExport 12 with BinDiff 8

Additional context
I have tried this with multiple statically compiled binaries and it seems to fail with the same error. Binaries that aren't statically linked do not have this error.

Build instructions failing when building from source due to binaryninjaapi error

Installing BinDiff from source according to building instructions in README currently fails due to binaryninjaapi git error.

Environment (please complete the following information):

  • OS and version: Ubuntu 22.04.3
  • Locale: NO Norwegian
  • Disassembler and version: IDA Pro 8.4

Building from Source (From README.md):

The following build dependencies are optional:

IDA Pro only: IDA SDK 8.0 or higher (unpack into deps/idasdk)
The general build steps are the same on Windows, Linux and macOS. The following shows the commands for Linux.

Download dependencies that won't be downloaded automatically:

mkdir -p build/out
git clone https://github.com/google/binexport build/binexport
unzip -q <path/to/idasdk_pro80.zip> -d build/idasdk

Next, configure the build directory and generate build files:

cmake -S . -B build/out -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=build/out \
  -DBINDIFF_BINEXPORT_DIR=build/binexport \
  "-DIdaSdk_ROOT_DIR=${PWD}build/idasdk"

Output error

user@laptop:~/bindiff-8$ cmake -S . -B build/out -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=build/out \
  -DBINDIFF_BINEXPORT_DIR=build/binexport \
  "-DIdaSdk_ROOT_DIR=${PWD}build/idasdk"
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX17
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX17 - Success
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX20
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX20 - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- 
-- 25.2.0
-- Performing Test protobuf_HAVE_LD_VERSION_SCRIPT
-- Performing Test protobuf_HAVE_LD_VERSION_SCRIPT - Success
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS - Success
-- Found Protobuf: protobuf::libprotobuf (found suitable version "4.25.2", minimum required is "3.14") 
[1/9] Creating directories for 'binaryninjaapi-populate'
[1/9] Performing download step (git clone) for 'binaryninjaapi-populate'
Cloning into 'binaryninjaapi-src'...
fatal: reference is not a tree: 0d57172be4fda0b7015db697b5353a12d03932af
CMake Error at binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/tmp/binaryninjaapi-populate-gitclone.cmake:40 (message):
  Failed to checkout tag: '0d57172be4fda0b7015db697b5353a12d03932af'


FAILED: binaryninjaapi-populate-prefix/src/binaryninjaapi-populate-stamp/binaryninjaapi-populate-download /home/user/bindiff-8/build/out/_deps/binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/src/binaryninjaapi-populate-stamp/binaryninjaapi-populate-download 
cd /home/user/bindiff-8/build/out/_deps && /usr/bin/cmake -P /home/user/bindiff-8/build/out/_deps/binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/tmp/binaryninjaapi-populate-gitclone.cmake && /usr/bin/cmake -E touch /home/user/bindiff-8/build/out/_deps/binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/src/binaryninjaapi-populate-stamp/binaryninjaapi-populate-download
ninja: build stopped: subcommand failed.

CMake Error at /usr/share/cmake-3.22/Modules/FetchContent.cmake:1087 (message):
  Build step for binaryninjaapi failed: 1
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/FetchContent.cmake:1216:EVAL:2 (__FetchContent_directPopulate)
  /usr/share/cmake-3.22/Modules/FetchContent.cmake:1216 (cmake_language)
  build/binexport/cmake/BinExportDeps.cmake:104 (FetchContent_Populate)
  build/binexport/CMakeLists.txt:42 (include)


-- Configuring incomplete, errors occurred!
See also "/home/user/bindiff-8/build/out/CMakeFiles/CMakeOutput.log".
See also "/home/user/bindiff-8/build/out/CMakeFiles/CMakeError.log".

Unable to build without IDA

Running cmake with -DBINEXPORT_ENABLE_IDAPRO=OFF results in the error:

CMake Error at build/binexport/cmake/FindIdaSdk.cmake:173 (find_path):
  Could not find IdaSdk_LIBPATH64 using the following files: libida64.so
Call Stack (most recent call first):
  cmake/BinDiffDeps.cmake:32 (find_package)
  CMakeLists.txt:49 (include)

Trying to add more filtering in IDA; Results seem counterintuivite

In advance: pardon my ignorance and noobishness.

Is your feature request related to a problem? Please describe.
Bindiff takes a lot of time to process binaries with large number of functions (currently, working with bins with 150k+ functions). Filtering by address might not always help.

Also, consider my use case: I am trying to preserve my work between patches in the program; alas, i am not really interested in what exactly has changed between patches;
I have names 20 functions that i'm interested in; and my goal in using bindiff (for the lack of more lightweight tool) is to identify those 20 functions in the new, patched binary: see if they are present, offsets have moved, etc etc.

Describe the solution you'd like
More filtering to be added to diff database filtered option of ida plugin.

Describe alternatives you've considered
I have tried to implement more filtering myself, and it almost works, but a little bit confusing with results.
differ.cc->SetupGraphsFromProto:

if (flow_graph_infos) {
      const auto address = flow_graph->GetEntryPointAddress();
      auto& info = (*flow_graph_infos)[address];
      info.address = address;
      auto name = flow_graph->GetName();
      if (skip_sub_funcs && starts_with(name, "sub_")) {
        //this is not a named function!
          flow_graph_infos->erase(address);
          flow_graph.reset();
          skipped++;
          continue;
          
      }

skip sub func parameter is passed from a gui checkbox added to diff database filtered dialog.
Reasoning: i have named 20 out of 150k functions, which means most of the functions i dont care about still have original ida naming (which is sub_address), so i try to skip those in the secondary (pre-patch) binary to not do unnecessary comparisons.
Adding this matches all 20 target functions correctly, while taking in total 4.5 minutes instead of 1.5 hours as usual.

What is confusing:

  1. all those sub_ functions are still in bindiff results (included into secondary unmatched window mostly)
  2. trying to add that same exact code into FilterFunctions (main_plugin.cc) gives 0 performance gain. (I havent tested for full 1.5 hours, but after 10 minutes of seeing "Performing Diff" window, i got tired of waiting):
if (entry_point < start || entry_point > end) {...}
else if (skip_sub)
{
    if (starts_with(flow_graph->GetName(), "sub_")
    {
        flow_graph_infos->erase(entry_point);
        delete flow_graph;
        flow_graphs->erase(it++);
    }
}
else
{
    ++it;
}

(I assumed that FilterFunctions would be more appropriate place to add the filtering).

Additional context
It would be nice to add some progress indication to performing diff wait_box, or maybe even a progress bar. In 1.5hr waiting timespan I start to question my sanity and think if its even doing anything (considering I also see only 2 running threads in IDA process at the time and taking 0.01% cpu i think it doesnt; but after 1.5hr automagically result always appears :D)

BinDiff crashes IDA when deleting more than one match

Steps to reproduce the problem

  1. Create diff between two IDBs
  2. Go to "matched functions tab"
  3. Select at least two matches (the more, the more reliable the crash -- preferably select all matches)
  4. Try deleting them via context menu

What is the expected behavior?

Not crash IDA

What went wrong?

Crashes IDA: https://gist.github.com/athre0z/00f5f4c883da33c0e377a177ed2413cd

What version of the product are you using? On what operating system?

Latest BinDiff 7 from website, IDA 8.1, macOS aarch64

Any other comments?

Hello Christian, hope you're well! :)

Ported from b/267647312

Manual matching does not trigger diff

Steps to reproduce the problem:

  1. Unmatch blocks.
  2. Match blocks.
  3. Observe that all text is different. See gif and image.

What is the expected behavior?
Text is diffed properly.

What went wrong?
Text is marked as different.

What version of the product are you using? On what operating system?
BinDiff 6, Win10.

177788191__32168413__191454

177788191__32168414__191454

Ported from b/177788191

Whether the tool support MachO file format

Overview of the Mach-O Executable Format

Mach-O is the native executable format of binaries in OS X and is the preferred format for shipping code. An executable format determines the order in which the code and data in a binary file are read into memory. The ordering of code and data has implications for memory usage and paging activity and thus directly affects the performance of your program.

A Mach-O binary is organized into segments. Each segment contains one or more sections. Code or data of different types goes into each section. Segments always start on a page boundary, but sections are not necessarily page-aligned. The size of a segment is measured by the number of bytes in all the sections it contains and rounded up to the next virtual memory page boundary. Thus, a segment is always a multiple of 4096 bytes, or 4 kilobytes, with 4096 bytes being the minimum size.

The segments and sections of a Mach-O executable are named according to their intended use. The convention for segment names is to use all-uppercase letters preceded by double underscores (for example, __TEXT); the convention for section names is to use all-lowercase letters preceded by double underscores (for example, __text).

There are several possible segments within a Mach-O executable, but only two of them are of interest in relation to performance: the __TEXT segment and the __DATA segment.

https://developer.apple.com/library/archive/documentation/Performance/Conceptual/CodeFootprint/Articles/MachOOverview.html

Importing symbols from a loaded diff in IDA is significantly slower than a diff performed within IDA

Steps to reproduce the problem:

  1. Load two large binaries and create IDBs
  2. Bindiff one of them against the other
  3. Import a couple of symbols and note the time it takes
  4. Save the diff results
  5. Restart IDA
  6. Load saved diff results
  7. Import the same number of symbols and again note the time it takes.

What is the expected behavior?

It should take about the same amount of time.

What went wrong?

I bindiffed bindiff (haha) against binexport and put breakpoints on calls to BinExport2::BinExport2 and google::protobuf::MessageLite::ParsePartialFromIstream. When working with a fresh diff performed within IDA bindiff does not load the binexport file for every symbol, but when working with loaded diff results binexport will load the binexport files for EACH symbol that's being ported.

What version of the product are you using? On what operating system? Bindiff 7, windows x64, IDA 7.6.210427

Any other comments? No

Ported from b/199001147

[BUG] Text in installer is undercut

Describe the bug
text in installer is undercut
msiexec_k99IdtiOzL

Environment (please complete the following information):

  • OS and version: Windows 10
  • Locale: English (UK)
  • Disassembler and version: v8 (from github)

Move functionality from plugins to a BinDiff server process

Is your feature request related to a problem? Please describe.

  • Diffing multiple times with the same primary from a disassembler is inefficient, as the export process needs to repeated each time
  • The IDA Pro plugin has a lot of functionality that has to be rewritten for each disassembler with few opportunities to share code (also C++ vs. Java in Ghidra)
  • The standalone UI also reimplements a lot of functionality of the IDA plugin, leading to subtle differences.

Describe the solution you'd like
Implement a local BinDiff RPC server using gRPC that get started on demand (similar to how Gradle/Bazel start build processes). The UI and other plugins can then start diffs (and the .BinExport will stay loaded in memory) and request flow graphs for view. This would help with implementing #16 as well. It would also enable BinDiff-as-a-service style deployments.

The plugins would become simpler and could be implemented in the repective disassembler's favoured scripting language (Python and PyQt for IDA Pro and Binary Ninja, Java Swing for Ghidra).

Describe alternatives you've considered
N/A

Additional context
It should be possible to implement this and adopt it gradually.

UI: Font-Family Should be configable

Describe the bug
Some characters are not displayed properly. Because bindiff trying to use Roboto Mono as the only font, However this font family doesn't support non-ASCII characters. Maybe font-family config can be add into config>main settings>general card.

To Reproduce
Just Open it and open any config window.

Screenshots
image

Environment (please complete the following information):

  • OS and version: Windows 10
  • Locale: zh-CN (Simplified Chinese)
  • Disassembler and version: IDA Pro 7.1

Additional context
Add any other context about the problem here.

Migrate away from CMake and use Bazel

Is your feature request related to a problem? Please describe.
Handling C++ dependencies in CMake is difficult. Bazel makes a lot easier, especially since BinDiff needs the OSS Google stack (Abseil, Protobuf, etc. but later maybe gRPC)

Describe the solution you'd like
Bazel has decent support for Windows, Linux and macOS and its build files are easier to maintain.

Describe alternatives you've considered
N/A

Additional context
This should first be investigated, but it's definitely feasible, as we are building most of BinDiff with the internal version of Bazel already (including IDA plugins and the UI).

automatic Symbol porting

Steps to reproduce the problem:

  1. Create golden database
  2. Created new database
  3. Try to port symbols from the command line

What is the expected behavior?
port symbols from the golden database to the new database

What went wrong?
Can't be done via command line

What version of the product are you using? On what operating system?
BinDiff 4/5 Ida 6.95/7.2

Any other comments? N/A

Ported from b/128820271

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.