lancern / asm2vec

An unofficial implementation of asm2vec as a standalone python package

Hi there 👋

impl Lancern {
    pub fn about() -> Self {
        Self {
            name: "Sirui Mu",
            email: "[email protected]",
            education: Education {
                bachelor: Degree("Beijing Institute of Technology", "Software Engineering", 2016..=2020),
                master: Degree("Tsinghua University", "Cybersecurity", 2020..=2023),
            },
        }
    }
}

impl Programmer for Lancern {
    fn topics(&self) -> Vec<&'static str> {
        vec!["Compilers", "Toolchain", "MLsys", "Storage Systems"]
    }

    fn languages(&self) -> Vec<&'static str> {
        vec!["C/C++", "C#", "CUDA", "Python", "Rust", "TypeScript", "Zig"]
    }
}

impl LinuxUser for Lancern {
    fn distros() -> Vec<&'static str> {
        vec!["openSUSE", "ArchLinux", "Fedora", "Ubuntu", "Debian"]
    }
}

impl FormerACMer for Lancern { }
impl FormerCTFer for Lancern { }

asm2vec's Issues

how did you get the assembly code from binary？

Hi, I want to know how did you get the assembly code from binary? What tools did you use? And can the tool mark the function boundary in assembly code? Thanks for your help!

Does the model need to be trained?

I am wondering if the model needs to be trained by a large dataset before using it to compare the similarities between 2 assembly functions?

Sigmoid Function is Incorrect

asm2vec/asm2vec/internal/training.py

Lines 187 to 188 in 6e975f0

	def _sigmoid(x: float) -> float:
	return 1 / (1 + np.exp(x))

should be

def _sigmoid(x: float) -> float:
    return 1 / (1 + np.exp(-x))

Only C？

Can I use the assembly code of Ethereum for identification? Is this model only trained in C language?

trigger some problems when running the training-estimating.py

I try to run training-estimating.py as following
python training-estimating.py

But it triggers some problems:
# of training functions: 3
# of estimating functions: 3
Training complete.
Traceback (most recent call last):
File "training-estimating.py", line 42, in
main()
File "training-estimating.py", line 29, in main
estimating_funcs_vec = list(map(lambda f: model.to_vec(f), estimating_funcs))
File "training-estimating.py", line 29, in
estimating_funcs_vec = list(map(lambda f: model.to_vec(f), estimating_funcs))
File "/home/bbge/Downloads/asm2vec-master/asm2vec/model.py", line 56, in to_vec
vf = estimate_repo.funcs()[0]
IndexError: list index out of range

My python version is 3.7.3.

Could you figure out how to deal with it?
Thanks!

Change ThreadPoolExecutor to ProcessPoolExecutor?

asm2vec/asm2vec/internal/repr.py

Line 105 in 6e975f0

executor = concurrent.futures.ThreadPoolExecutor(max_workers=jobs)

asm2vec/asm2vec/internal/training.py

Line 286 in 6e975f0

executor = concurrent.futures.ThreadPoolExecutor(max_workers=context.params().jobs)

Using thread to handle CPU-bound tasks in python may not be helpful due to the issue of GIL.

something wrong with the example

vf = estimate_repo.funcs()[0]
IndexError: list index out of range

When I try to run the example. Here comes a bug.

I can’t specify the value of "d"

When I set d = 256 or other values, it will cause a bug.

Naming Conflict with Python Standard Library Module 'logging'

Problem:
When running any of the files that utilize asm2vec.logging, an error occurs due to a naming conflict with the standard library module logging. (This will happen if you exported the python path to /path/to/asm2vec)

Solution:
To resolve this, I renamed the asm2vec.logging module to a different name, and I updated the calls within all related files accordingly. This was necessary to avoid conflicts with the standard logging module from the Python standard library.

Details:

Renamed: /asm2vec/asm2vec/logging.py
New Name: logingue
Adjusted calls in all relevant files

Reason:
The naming conflict arises because the standard library has a module named logging. By renaming the specific module within the asm2vec package, I was able to avoid this conflict.

Note:
Changing the module name is not as complex as it may seem, as the interpreter provides information about the file where the module was called. Additionally, aliases are used when importing modules, reducing the need to modify the module name at every function call.

Here's the error i got:

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/home/ubuntu/asm2vec/asm2vec/logging.py", line 4, in <module>

    def asm2vec_logger() -> logging.Logger:

AttributeError: partially initialized module 'logging' has no attribute 'Logger' (most likely due to a circular import)

Error in sys.excepthook:

Traceback (most recent call last):

  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook

    from apport.fileutils import likely_packaged, get_recent_crashes

  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>

    from apport.report import Report

  File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in <module>

    import apport.fileutils

  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 28, in <module>

    from apport.packaging_impl import impl as packaging

  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>

    import apt

  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 26, in <module>

    from apt.package import Package as Package, Version as Version

  File "/usr/lib/python3/dist-packages/apt/package.py", line 24, in <module>

    import logging

  File "/home/ubuntu/asm2vec/asm2vec/logging.py", line 4, in <module>

    def asm2vec_logger() -> logging.Logger:

AttributeError: partially initialized module 'logging' has no attribute 'Logger' (most likely due to a circular import)



Original exception was:

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/home/ubuntu/asm2vec/asm2vec/logging.py", line 4, in <module>

    def asm2vec_logger() -> logging.Logger:

AttributeError: partially initialized module 'logging' has no attribute 'Logger' (most likely due to a circular import)

This doesn't work on the generate .asm files from IDA?

Hello, is there anyway to make this work on the generated .asm files from IDA? it seems to expect a specific type of input, so considering IDA's asm files have many information in them it keeps causing exceptions and doesn't work?