GithubHelp home page GithubHelp logo

delvelabs / batea Goto Github PK

View Code? Open in Web Editor NEW
296.0 19.0 47.0 74 KB

AI-based, context-driven network device ranking

Home Page: https://batea.delvesecurity.com/

License: GNU General Public License v2.0

Python 100.00%

batea's People

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

bvanheu mjgrey hotelzululima olivierh59500 y0d4a gavz kenthunt inspectordidi khasmek blue-infosec jr69ss jermainlaforce jakuta-tech pixelza tundeogunkoya kaisaryousuf trendingtechnology 5l1v3r1 thecarbonista polling-repo-continua sergeolivierp kgroveshok flaviupopescu marciopocebon sashka3076 rajivraj moisestapia fhgzs raystyle jorson-chen j4ckzh0u segxi dasgp mzpqnxow alexxxisanchezz natecrisler siberavci nitbx aviau powerpress fjm1995 dekoder flared conquering-coding l1kw1d powned mybusybeeinc

batea's Issues

Issues installing batea

When i tried to install batea i got the below error and I am unsure why and I am running it on Darwin HQSML-1689616 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64

python3 setup.py sdist; pip3 install -r requirements.txt; pip3 install -e .;
running sdist
running egg_info
error: [Errno 13] Permission denied
Requirement already satisfied: defusedxml==0.6.0 in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (0.6.0)
Requirement already satisfied: numpy==1.17.2 in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (1.17.2)
Requirement already satisfied: click==6.7 in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (6.7)
Collecting scikit-learn==0.21.3
  Using cached scikit-learn-0.21.3.tar.gz (12.2 MB)
Processing /Users/gbiago909/Library/Caches/pip/wheels/5f/a4/f5/58f4823fa59f34b12e2ad34a5f05d443346f42aa486a4df1aa/pandas-0.25.0-cp38-cp38-macosx_10_15_x86_64.whl
Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.8/site-packages (from scikit-learn==0.21.3->-r requirements.txt (line 4)) (1.5.2)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.8/site-packages (from scikit-learn==0.21.3->-r requirements.txt (line 4)) (0.16.0)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.8/site-packages (from pandas==0.25.0->-r requirements.txt (line 5)) (2020.1)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.8/site-packages (from pandas==0.25.0->-r requirements.txt (line 5)) (2.8.1)
Requirement already satisfied: six>=1.5 in /usr/local/Cellar/protobuf/3.12.4/libexec/lib/python3.8/site-packages (from python-dateutil>=2.6.1->pandas==0.25.0->-r requirements.txt (line 5)) (1.15.0)
Building wheels for collected packages: scikit-learn
  Building wheel for scikit-learn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/opt/[email protected]/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py'"'"'; __file__='"'"'/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-wheel-1_qd4w7j
       cwd: /private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/
  Complete output (28 lines):
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py", line 290, in <module>
      setup_package()
    File "/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py", line 286, in setup_package
      setup(**metadata)
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
      config = configuration()
    File "/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py", line 174, in configuration
      config.add_subpackage('sklearn')
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
      config_list = self.get_subpackage(subpackage_name, subpackage_path,
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
      config = self._get_configuration_from_setup_py(
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/setup.py", line 62, in configuration
      config.add_subpackage('utils')
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
      config_list = self.get_subpackage(subpackage_name, subpackage_path,
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
      config = self._get_configuration_from_setup_py(
    File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/utils/setup.py", line 8, in configuration
      from Cython import Tempita
  ModuleNotFoundError: No module named 'Cython'
  ----------------------------------------
  ERROR: Failed building wheel for scikit-learn
  Running setup.py clean for scikit-learn
Failed to build scikit-learn
Installing collected packages: scikit-learn, pandas
    Running setup.py install for scikit-learn ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/opt/[email protected]/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py'"'"'; __file__='"'"'/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-record-t3su3ll1/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/scikit-learn
         cwd: /private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/
    Complete output (28 lines):
    Partial import of sklearn during the build process.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py", line 290, in <module>
        setup_package()
      File "/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py", line 286, in setup_package
        setup(**metadata)
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py", line 174, in configuration
        config.add_subpackage('sklearn')
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/setup.py", line 62, in configuration
        config.add_subpackage('utils')
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/usr/local/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/opt/[email protected]/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py'"'"'; __file__='"'"'/private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-install-vxaj13m8/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/j5/t3kln5r959vc5bfr4hp20vmh0000gn/T/pip-record-t3su3ll1/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/scikit-learn Check the logs for full command output.
Obtaining file:///opt/batea
Requirement already satisfied: defusedxml==0.6.0 in /usr/local/lib/python3.8/site-packages (from batea==0.0.1) (0.6.0)
Installing collected packages: batea
  Attempting uninstall: batea
    Found existing installation: batea 0.0.1
    Uninstalling batea-0.0.1:
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.8/site-packages/easy-install.pth'
Consider using the `--user` option or check the permissions.

Output from when i tru to run it

Traceback (most recent call last):
  File "/usr/local/bin/batea", line 33, in <module>
    sys.exit(load_entry_point('batea==0.0.1', 'console_scripts', 'batea')())
  File "/usr/local/bin/batea", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/usr/local/lib/python3.8/site-packages/batea-0.0.1-py3.8.egg/batea/__init__.py", line 18, in <module>
ModuleNotFoundError: No module named 'batea.core'

I am having the same errors on my ubuntu server which i retuned to be like kali Linux kali 4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020 x86_64 GNU/Linux

Unusual ports feature

Thoughts on having a feature about unusual or hacker related ports? I was thinking about ports like 4444, 1337, 31337. The first is Metasploit's default port while the second two are "hacker" port numbers. I think these would be worth investigating if I saw these in a scan since they aren't normally used.

Support MAC Address in training data

Feature idea:
Using MAC address might help to reveal faked random MAC or vendors that are not supposed to show up in your network or are obscure IoT embedded devices.

ValueError: numpy.ndarray size changed

When installing Batea in an venv I get the following error if I try to execute it:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

AttributeError: 'NoneType' object has no attribute 'findall' 

Traceback (most recent call last):
  File "/usr/local/bin/batea", line 11, in <module>
    load_entry_point('batea', 'console_scripts', 'batea')()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/Documents/Projects/batea/batea/__main__.py", line 51, in main
    report.hosts.extend([host for host in xml_parser.load_hosts(file)])
  File "/Users/user/Documents/Projects/batea/batea/__main__.py", line 51, in <listcomp>
    report.hosts.extend([host for host in xml_parser.load_hosts(file)])
  File "/Users/user/Documents/Projects/batea/batea/core/nmap_parser.py", line 29, in load_hosts
    host = self._generate_host(child)
  File "/Users/user/Documents/Projects/batea/batea/core/nmap_parser.py", line 37, in _generate_host
    ports=self._find_ports(subtree))
  File "/Users/user/Documents/Projects/batea/batea/core/nmap_parser.py", line 52, in _find_ports
    for port in host.find("ports").findall("port"):
AttributeError: 'NoneType' object has no attribute 'findall'

Fails to parse nmap output files 

❯ batea -h
Usage: batea [OPTIONS] [NMAP_REPORTS]...

  Context-driven asset ranking based using anomaly detection

Options:
  -c, --read-csv FILENAME
  -x, --read-xml FILENAME
  -n, --n-output INTEGER
  -A, --output-all
  -L, --load-model FILENAME
  -D, --dump-model FILENAME
  -f, --input-format TEXT
  -v, --verbose
  -oM, --output-matrix FILENAME
  -h, --help                     Show this message and exit.
❯ batea -v -x output_3.xml
/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/sklearn/ensemble/iforest.py:478: RuntimeWarning: invalid value encounte                                                                                                                                              red in true_divide
  -depths
/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/sklearn/ensemble/iforest.py:478: RuntimeWarning: invalid value encounte                                                                                                                                              red in true_divide
  -depths
{
    "report_info": [
        {
            "number_of_hosts": 1,
            "features": [
                "ip_octet_0",
                "ip_octet_1",
                "ip_octet_2",
                "ip_octet_3",
                "port_count",
                "open_port_count",
                "low_port_count",
                "tcp_port_count",
                "named_service_count",
                "software_banner_count",
                "max_banner_length",
                "is_windows",
                "is_linux",
                "http_server_count",
                "database_count",
                "windows_domain_admin_count",
                "windows_domain_member_count",
                "port_entropy",
                "hostname_length",
                "hostname_entropy"
            ]
        }
    ],
    "host_info": [
        {
            "rank": "1",
            "host": "x.x.x.113",
            "score": NaN,
            "hostname": null,
            "os": {
                "vendor": "Linux",
                "family": "Linux",
                "type": "general purpose",
                "name": "Linux 3.10 - 4.8",
                "accuracy": 96
            },
            "features": {
                "ip_octet_0": x,
                "ip_octet_1": x,
                "ip_octet_2": x,
                "ip_octet_3": 113.0,
                "port_count": 3.0,
                "open_port_count": 3.0,
                "low_port_count": 3.0,
                "tcp_port_count": 3.0,
                "named_service_count": 3.0,
                "software_banner_count": 3.0,
                "max_banner_length": 7.0,
                "is_windows": 0.0,
                "is_linux": 1.0,
                "http_server_count": 2.0,
                "database_count": 0.0,
                "windows_domain_admin_count": 0.0,
                "windows_domain_member_count": 0.0,
                "port_entropy": 1.584962500721156,
                "hostname_length": 0.0,
                "hostname_entropy": 0.0
            }
        }
    ]
}
❯ batea -v -x output.xml
Traceback (most recent call last):
  File "/home/drdinosaur/.pyenv/versions/3.8.0/bin/batea", line 11, in <module>
    load_entry_point('batea', 'console_scripts', 'batea')()
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/drdinosaur/batea/batea/__main__.py", line 60, in main
    report.hosts.extend([host for host in xml_parser.load_hosts(file)])
  File "/home/drdinosaur/batea/batea/__main__.py", line 60, in <listcomp>
    report.hosts.extend([host for host in xml_parser.load_hosts(file)])
  File "/home/drdinosaur/batea/batea/core/nmap_parser.py", line 29, in load_hosts
    host = self._generate_host(child)
  File "/home/drdinosaur/batea/batea/core/nmap_parser.py", line 37, in _generate_host
    ports=self._find_ports(subtree))
  File "/home/drdinosaur/batea/batea/core/nmap_parser.py", line 52, in _find_ports
    for port in host.find("ports").findall("port"):
AttributeError: 'NoneType' object has no attribute 'findall'
❯ batea -v -x output_2.xml
Traceback (most recent call last):
  File "/home/drdinosaur/.pyenv/versions/3.8.0/bin/batea", line 11, in <module>
    load_entry_point('batea', 'console_scripts', 'batea')()
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/drdinosaur/.pyenv/versions/3.8.0/lib/python3.8/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/drdinosaur/batea/batea/__main__.py", line 60, in main
    report.hosts.extend([host for host in xml_parser.load_hosts(file)])
  File "/home/drdinosaur/batea/batea/__main__.py", line 60, in <listcomp>
    report.hosts.extend([host for host in xml_parser.load_hosts(file)])
  File "/home/drdinosaur/batea/batea/core/nmap_parser.py", line 29, in load_hosts
    host = self._generate_host(child)
  File "/home/drdinosaur/batea/batea/core/nmap_parser.py", line 37, in _generate_host
    ports=self._find_ports(subtree))
  File "/home/drdinosaur/batea/batea/core/nmap_parser.py", line 52, in _find_ports
    for port in host.find("ports").findall("port"):
AttributeError: 'NoneType' object has no attribute 'findall'

The tool is able to handle an output with one host, but errors out with 9 or 10 host files.

Error running batea

Hi, i have just tried to install and run the project, with or without sudo, and I am getting errors from numpy. I am running kali

`┌──(kali㉿kali)-[~/git/batea]
└─$ batea -v output.xml
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
File "/home/kali/.local/bin/batea", line 33, in
sys.exit(load_entry_point('batea', 'console_scripts', 'batea')())
File "/home/kali/.local/bin/batea", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 86, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/kali/git/batea/batea/init.py", line 18, in
from .core.nmap_parser import NmapReportParser
File "/home/kali/git/batea/batea/core/init.py", line 22, in
from .model import BateaModel
File "/home/kali/git/batea/batea/core/model.py", line 18, in
from sklearn.ensemble import IsolationForest
File "/home/kali/.local/lib/python3.9/site-packages/sklearn/init.py", line 80, in
from .base import clone
File "/home/kali/.local/lib/python3.9/site-packages/sklearn/base.py", line 21, in
from .utils import _IS_32BIT
File "/home/kali/.local/lib/python3.9/site-packages/sklearn/utils/init.py", line 20, in
from scipy.sparse import issparse
File "/usr/lib/python3/dist-packages/scipy/sparse/init.py", line 228, in
from .csr import *
File "/usr/lib/python3/dist-packages/scipy/sparse/csr.py", line 10, in
from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks,
ImportError: numpy.core.multiarray failed to import

┌──(kali㉿kali)-[~/git/batea]
└─$ cd ..

┌──(kali㉿kali)-[~/git]
└─$ rm -rf batea

┌──(kali㉿kali)-[~/git]
└─$ git clone https://github.com/delvelabs/batea.git
Cloning into 'batea'...
remote: Enumerating objects: 258, done.
remote: Counting objects: 100% (68/68), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 258 (delta 59), reused 45 (delta 45), pack-reused 190
Receiving objects: 100% (258/258), 73.92 KiB | 764.00 KiB/s, done.
Resolving deltas: 100% (144/144), done.

┌──(kali㉿kali)-[~/git]
└─$ cd batea

┌──(kali㉿kali)-[~/git/batea]
└─$ sudo python3 setup.py sdist
[sudo] password for kali:
running sdist
running egg_info
creating batea.egg-info
writing batea.egg-info/PKG-INFO
writing dependency_links to batea.egg-info/dependency_links.txt
writing entry points to batea.egg-info/entry_points.txt
writing requirements to batea.egg-info/requires.txt
writing top-level names to batea.egg-info/top_level.txt
writing manifest file 'batea.egg-info/SOURCES.txt'
reading manifest file 'batea.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'batea.egg-info/SOURCES.txt'
running check
creating batea-0.0.1
creating batea-0.0.1/batea
creating batea-0.0.1/batea.egg-info
copying files to batea-0.0.1...
copying LICENSE.md -> batea-0.0.1
copying README.md -> batea-0.0.1
copying setup.cfg -> batea-0.0.1
copying setup.py -> batea-0.0.1
copying batea/init.py -> batea-0.0.1/batea
copying batea/main.py -> batea-0.0.1/batea
copying batea/version.py -> batea-0.0.1/batea
copying batea.egg-info/PKG-INFO -> batea-0.0.1/batea.egg-info
copying batea.egg-info/SOURCES.txt -> batea-0.0.1/batea.egg-info
copying batea.egg-info/dependency_links.txt -> batea-0.0.1/batea.egg-info
copying batea.egg-info/entry_points.txt -> batea-0.0.1/batea.egg-info
copying batea.egg-info/requires.txt -> batea-0.0.1/batea.egg-info
copying batea.egg-info/top_level.txt -> batea-0.0.1/batea.egg-info
Writing batea-0.0.1/setup.cfg
creating dist
Creating tar archive
removing 'batea-0.0.1' (and everything under it)

┌──(kali㉿kali)-[~/git/batea]
└─$ sudo pip3 install -r requirements.txt
Collecting defusedxml==0.6.0
Downloading defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
Collecting numpy==1.19.4
Downloading numpy-1.19.4-cp39-cp39-manylinux2010_x86_64.whl (14.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.5/14.5 MB 10.2 MB/s eta 0:00:00
Collecting click==7.1.2
Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.8/82.8 KB 15.0 MB/s eta 0:00:00
Collecting scikit-learn==0.23.2
Downloading scikit-learn-0.23.2.tar.gz (7.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.2/7.2 MB 9.4 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting pandas==1.1.4
Downloading pandas-1.1.4-cp39-cp39-manylinux1_x86_64.whl (9.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.3/9.3 MB 10.1 MB/s eta 0:00:00
Collecting joblib>=0.11
Downloading joblib-1.1.0-py2.py3-none-any.whl (306 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.0/307.0 KB 7.9 MB/s eta 0:00:00
Collecting threadpoolctl>=2.0.0
Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: scipy>=0.19.1 in /usr/lib/python3/dist-packages (from scikit-learn==0.23.2->-r requirements.txt (line 4)) (1.7.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/lib/python3/dist-packages (from pandas==1.1.4->-r requirements.txt (line 5)) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in /usr/lib/python3/dist-packages (from pandas==1.1.4->-r requirements.txt (line 5)) (2022.1)
Building wheels for collected packages: scikit-learn
Building wheel for scikit-learn (pyproject.toml) ... done
Created wheel for scikit-learn: filename=scikit_learn-0.23.2-cp39-cp39-linux_x86_64.whl size=21712114 sha256=931033a0fc5847f2cf85d6f29d0aa88d849cf99df627e5b4a514a7316ececd6b
Stored in directory: /root/.cache/pip/wheels/5e/74/24/7e235ccf01765c0daa089c98cc823e9dc1383da5fe0ed7e224
Successfully built scikit-learn
Installing collected packages: threadpoolctl, numpy, joblib, defusedxml, click, scikit-learn, pandas
Attempting uninstall: numpy
Found existing installation: numpy 1.21.5
Not uninstalling numpy at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'numpy'. No files were found to uninstall.
Attempting uninstall: defusedxml
Found existing installation: defusedxml 0.7.1
Not uninstalling defusedxml at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'defusedxml'. No files were found to uninstall.
Attempting uninstall: click
Found existing installation: click 8.0.3
Not uninstalling click at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'click'. No files were found to uninstall.
Attempting uninstall: pandas
Found existing installation: pandas 1.3.5
Not uninstalling pandas at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'pandas'. No files were found to uninstall.
Successfully installed click-7.1.2 defusedxml-0.6.0 joblib-1.1.0 numpy-1.19.4 pandas-1.1.4 scikit-learn-0.23.2 threadpoolctl-3.1.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

┌──(kali㉿kali)-[~/git/batea]
└─$ sudo pip3 install -e .
Obtaining file:///home/kali/git/batea
Preparing metadata (setup.py) ... done
Requirement already satisfied: defusedxml==0.6.0 in /usr/local/lib/python3.9/dist-packages (from batea==0.0.1) (0.6.0)
Installing collected packages: batea
Running setup.py develop for batea
Successfully installed batea-0.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

┌──(kali㉿kali)-[~/git/batea]
└─$ cp ../../output.xml .

┌──(kali㉿kali)-[~/git/batea]
└─$ sudo batea -v output.xml
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Traceback (most recent call last):
File "/usr/local/bin/batea", line 33, in
sys.exit(load_entry_point('batea', 'console_scripts', 'batea')())
File "/usr/local/bin/batea", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 86, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/kali/git/batea/batea/init.py", line 18, in
from .core.nmap_parser import NmapReportParser
File "/home/kali/git/batea/batea/core/init.py", line 22, in
from .model import BateaModel
File "/home/kali/git/batea/batea/core/model.py", line 18, in
from sklearn.ensemble import IsolationForest
File "/usr/local/lib/python3.9/dist-packages/sklearn/init.py", line 80, in
from .base import clone
File "/usr/local/lib/python3.9/dist-packages/sklearn/base.py", line 21, in
from .utils import _IS_32BIT
File "/usr/local/lib/python3.9/dist-packages/sklearn/utils/init.py", line 20, in
from scipy.sparse import issparse
File "/usr/lib/python3/dist-packages/scipy/sparse/init.py", line 228, in
from .csr import *
File "/usr/lib/python3/dist-packages/scipy/sparse/csr.py", line 10, in
from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks,
ImportError: numpy.core.multiarray failed to import

┌──(kali㉿kali)-[~/git/batea]
└─$ lsb_release -r
Release: 2022.1

┌──(kali㉿kali)-[~/git/batea]
└─$ lsb_release -ra
No LSB modules are available.
Distributor ID: Kali
Description: Kali GNU/Linux Rolling
Release: 2022.1
Codename: kali-rolling
`

It looks like requirements process does not uninstall numpy version and install the good one, right?

but, if I check installed dependencies with required ones, I think I have no problem:

`┌──(kali㉿kali)-[~/git/batea]
└─$ cat requirements.txt
defusedxml==0.6.0
numpy==1.19.4
click==7.1.2
scikit-learn==0.23.2
pandas==1.1.4

┌──(kali㉿kali)-[~/git/batea]
└─$ python3 -m pip show numpy
Name: numpy
Version: 1.19.4
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email:
License: BSD
Location: /home/kali/.local/lib/python3.9/site-packages
Requires:
Required-by: ImageHash, pandas, PyWavelets, scikit-learn

┌──(kali㉿kali)-[~/git/batea]
└─$ python3 -m pip show defusedxml
Name: defusedxml
Version: 0.6.0
Summary: XML bomb protection for Python stdlib modules
Home-page: https://github.com/tiran/defusedxml
Author: Christian Heimes
Author-email: [email protected]
License: PSFL
Location: /home/kali/.local/lib/python3.9/site-packages
Requires:
Required-by: batea

┌──(kali㉿kali)-[~/git/batea]
└─$ python3 -m pip show click
Name: click
Version: 7.1.2
Summary: Composable command line interface toolkit
Home-page: https://palletsprojects.com/p/click/
Author:
Author-email:
License: BSD-3-Clause
Location: /home/kali/.local/lib/python3.9/site-packages
Requires:
Required-by: uvicorn

┌──(kali㉿kali)-[~/git/batea]
└─$ python3 -m pip show scikit-learn
Name: scikit-learn
Version: 0.23.2
Summary: A set of python modules for machine learning and data mining
Home-page: http://scikit-learn.org
Author:
Author-email:
License: new BSD
Location: /home/kali/.local/lib/python3.9/site-packages
Requires: joblib, numpy, scipy, threadpoolctl
Required-by:

┌──(kali㉿kali)-[~/git/batea]
└─$ python3 -m pip show pandas
Name: pandas
Version: 1.1.4
Summary: Powerful data structures for data analysis, time series, and statistics
Home-page: https://pandas.pydata.org
Author:
Author-email:
License: BSD
Location: /home/kali/.local/lib/python3.9/site-packages
Requires: numpy, python-dateutil, pytz
Required-by:
`

so, could you help me please?

No longer works

Hello this is a great tool but it no longer works , can you please test in and update the readme if possible by any chance ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.