e-m-b-a / embark Goto Github PK

View Code? Open in Web Editor NEW

296.0 7.0 41.0 53.68 MB

EMBArk - The firmware security scanning environment

Home Page: https://www.securefirmware.de

License: MIT License

Dockerfile 0.35% Python 46.11% Shell 17.29% CSS 4.13% JavaScript 12.02% HTML 20.10%

linux iot security firmware hacking embedded-systems penetration-testing embedded-linux security-tools pentesting

embark's Introduction

EMBArk - The firmware security scanning environment

About

EMBArk is a tool for centralized firmware analyzing. Scan, Identify, Track, Report. Built to be simple but powerful. It's the web-based enterprise interface for the firmware security scanner EMBA. It is developed to provide the firmware security analyzer EMBA as a service with feature-packed UI and to improve accessibility to the firmware scanning backend EMBA regardless of the system and operating system. Furthermore, EMBArk improves the data provision by aggregating the various scanning results in an aggregated management dashboard.

Important

The new Tracking application currently doesn't work offline.
The device, Vendor and version tracking aren't fully integrated into all dashboards yet.
the only supported OS at this time is Ubuntu LTS (22.04)

Quick-Start

git clone https://github.com/e-m-b-a/embark.git; cd embark; sudo ./installer.sh -d

sudo ./run-server.sh [-a <IP/HOSTNAME>] [-h]

Automated setup on Ubuntu 22.04 LTS

Checkout the repository (e.g. git clone https://github.com/e-m-b-a/embark.git)
Change directory to root of the repository i.e cd embark
Run sudo ./installer.sh -d to run the default installation.
Further details are available in the wiki (See installation)

EMBArk is using the firmware security scanner EMBA as backend. You can take a look at the EMBA repository and get more information about usage of EMBA in the wiki. Additionally you should check the EMBArk wiki.

Start Server

To start the EMBArk-Server simply run $ sudo ./run-server.sh. This starts the http-Server on 0.0.0.0:80

Note: The default server name is "embark.local" and has to be resolved via host files or a DNS-server.
If you want to query the server using an IP or other hostname please use the -a option. (multiple inputs supported)

Upgrading

Use the export-DB.sh to back up your database
Just pull and restart
Usually, upgrading does NOT require a reinstallation (reinstalling will delete all files and the database completely)

Developer

For developers, we recommend simply using: sudo ./installer.sh -F and the ./dev-tools/debug-server-start.sh script.

Get involved

The IoT is growing, the development is ongoing, and there are many new features that we want to add. We welcome pull requests and bug reports or feature requests via issues on GitHub.

Acknowledgement

This project was originally initiated by Siemens Energy as AMOS project in cooperation with the FAU.

See also the original EMBArk AMOS project and AMOS.

embark's People

Contributors

Stargazers

Watchers

embark's Issues

"Firmware scan status" view missing

Describe the bug
Couldn't find "Firmware scan status" view in the dashboard

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh -a 192.168.1.20

Expected behavior

Once after the firmware uploaded and option selected to analyze, I could find the option to view the scan status.

Screenshots

Desktop (please complete the following information):

OS: Kali Linux 2022.03

Additional context

Upon selecting the option "Progress" in the right-side panel I could only see the option "Select and stop Analysis" but I couldn't get an option anywhere in the dashboard as mentioned in the image located https://github.com/e-m-b-a/embark/wiki/Web-interface#firmware-scan-status
All I could do is that I can choose the firmware which I uploaded in the drop-down under "Select and stop Analysis" and a STOP button.
Am I missing something during the installation, or any configuration needs to be changed to view the scan status? Please advise.

Final report header fix

Hello Benedikt,

I checked today the upload and reporting in EMBArk and I have these info:

I added these parameters:

and I got this back:

The final report would be important I think because the user would store only the html pages of the scans.
Could you check it on your side please? Thank you.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh
Use any firmware
See the issues.

Expected behavior
Final HTML report should contain the same version number and notes of the actual scan.

Desktop (please complete the following information):

OS: [Kali Linux 2022.3]

emba_runs does not exist

Docker Interface - Device "emba_runs" does not exist.
Missing Docker-Interface - check your installation
Trying to auto-maintain the docker interface ...
Device "emba_runs" does not exist.
Use $systemctl restart NetworkManager docker or reset the docker interface manually ($ docker network rm emba_runs)
cve-search - testing

使用ifconfig查询，网卡是docker0.执行installer.sh -d的时候会自动查找emba_runs，找不到的情况下就报上面的错误。这个问题怎么解决。

emba_updater without branch - question

Describe the bug
I checked the emba_updater script on Kali (using emba) and on Ubuntu Server (using EMBArk) and on Kali the main repository is used:

but on Ubuntu Server I have to define the branch manually:

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk update script: sudo ./emba_updater
See git messages.

Expected behavior
EMBArk update script should work without manual hit configuration.

Screenshots
See above.

Desktop (please complete the following information):

OS: Kali Linux 2022.4 and Ubuntu Server 22.04.1

Toggle Tests for each scan

Thanks for your great work on embark! Came across this some time and follow your development. While working with Emba I come across several things that would be great to be implemented, I will add them as feature requests for the projects, so it can be discussed if they are valid or not.

Feature request:
It would be great to have the possibility to disable individual tests.

For example the grepit tests takes a lot of time but delivers only marginal results. It would be great to customize the scan as much as possible.

There is already an "Expert Mode" when starting the scan, so it would be good if all modules / scans could be toggled on / off here.

I think in uploader/tests.py or uploader/models.py the command line options for emba are defined, have not looked into detail if this can be modified there.

Getting started with Embark

I want to try this tool, but i got stuck on getting it up and running.

I Created a hyper-VM, and installed linux kali on it.
I cloned the embark repository
And I started the installer from whitin the embark directory:
sudo ./installer.sh -F
The next step should be registering a new user, but there's nothing listening on port 80
What am I missing here?

Scan progress frame height

In EMBArk the scan progress pages shows the actual modules but the frame is very small, it would be better in bigger size:

I know it is nice to have but it would be easier to follow and understand what is happening at the scan.

Delete button on report page

Is your feature request related to a problem? Please describe.
EMBArk's report page displays all the successful and failed scans. If I reexecute a test it will provide new line on the page. I think the user should be able to delete manually the unnecessary lines even if it was good or failed. The rescan/delete button just allows the user to delete the uploaded FW file which is perfect. Deleting the uploaded FW should leave the test result on the report page like works correctly.

Describe the solution you'd like
A new delete option for the scan result may allow the user to maintain the report page on long term.

Additional context

pip-apt issue

EMBA-version requires update

Issue:

desc:
The EMBA version is behind main

Entropy value is always zero in detailed view

Describe the bug
The final report contains enthropy graph and value but the detailed view in EMBArk shows only 0.

so the graph on the main page is missing:

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh
Use the firmware available here: anything
Do a scan
Check enthropy on detailed view page.

Desktop (please complete the following information):

OS: Ubuntu Server 22.04.1

Module dependencies in Expert Mode

Is your feature request related to a problem? Please describe.

I tested all modules one by one and found some dependencies.
S13 requires S12 and S14 requires S12 and S13:

S20 needs S115:

S25 uses S24:

S109 requires S108:

S116 wanted to use S115:

Each modul can be selected one by one and if the user does not have the knowledge of exactly which module does what, they cannot effectively select for the test.

Describe the solution you'd like

It would be nice to have a way to display the dependencies in the list if someone is using the expert mode:

Describe alternatives you've considered

I think the framed solution would show the related modules well, but if someone selects one that has a dependency it could also be selected automatically.

Additional context

If you choose expert mode, no module is selected by default. I think that in practise we often need almost all modules and want to leave some out, so the list of modules should be all selected except for S120_cwe_checker, and then we should just manually select the one or ones we don't want to use in the actual test execution. Functions such as select all, deselect all and reverse selection could also help the user to manage the list.
If the module_blacklist.txt file exists then EMBArk could take into account the disabled modules and in this case gray out them on the page so that they cannot be selected. At the moment even though they are disabled in the text file, they still run when manually selected.

I think the following modules are missing from the page but the scripts are in modules folder:

S17_apk_check.sh
S26_kernel_vuln_verifier.sh
S36_lighttpd.sh

Request Entity Too Large

Describe the bug

While uploading "large" files the upload UI is not very clear on why an upload is failing.
Doing some debugging in the web console I was able to see "Request Entity Too Large".
Pressing the upload button a few times also causes the progress bar to go.. weird

is there a setting to allow larger files?
is there also a setting to allow to queue more analysis ?

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (using EMBABox)
Start EMBArk: sudo ./run-server.sh
Use the firmware available here:430 MB file (ota update)
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. Kali Linux 2022.01]

Additional context
Add any other context about the problem here.

Not sure but I also got this message (perhaps a race condition while uploading?)

2022-04-19 10:21:09,597 INFO     Listening on TCP address 0.0.0.0:8001                                    
Error notifying listener                             
Traceback (most recent call last):                   
  File "/home/vagrant/embark/.venv/lib/python3.9/site-packages/apscheduler/schedulers/base.py", line 836, in _dispatch_event
    cb(event)                                        
  File "/home/vagrant/embark/.venv/lib/python3.9/site-packages/django_apscheduler/jobstores.py", line 100, in handle_execution_event
    job_execution = DjangoJobExecution.atomic_update_or_create(                                           
  File "/home/vagrant/embark/.venv/lib/python3.9/site-packages/django_apscheduler/util.py", line 99, in func_wrapper
    result = func(*args, **kwargs)                   
  File "/home/vagrant/embark/.venv/lib/python3.9/site-packages/django_apscheduler/models.py", line 165, in atomic_update_or_create
    job_execution = DjangoJobExecution.objects.select_for_update().get(                                   
  File "/home/vagrant/embark/.venv/lib/python3.9/site-packages/django/db/models/query.py", line 443, in get
    raise self.model.MultipleObjectsReturned(                                                             
django_apscheduler.models.DjangoJobExecution.MultipleObjectsReturned: get() returned more than one DjangoJobExecution -- it returned 2!

Firmware deletion does not work

I created a new Kali environment and installed an EMBArk successfully. I did some FW scans and I wanted to delete a firmware image from EMBArk. Delete -> OK and EMBArk logged out and "Account successfully deleted." message displayed.
I made the registration with the same user / password and I was ablo to log back in and I got the previous results but the selected FW image was there.
The firmware image deletion destroys the registration and does not work as expected.

I reproduced the same scenarion on a different environment and I got the same result.

VMware Workstation Pro
Kali 2022.1
Actual EMBArk

Kali Linux 2022.01 is out

See here https://www.kali.org/blog/kali-linux-2022-1-release/

Need to wait till e-m-b-a/emba#220 is finished

Testcases:

Installation
User add/remove/pw change/login
Login
EMBA test with default settings (check scan dashboard, main dashboard, EMBA report)
EMBA test with full scan settings (check scan dashboard, main dashboard, EMBA report)
Dashboard access without login
dev mode

Firmware delete

Describe the bug
If I delete an uploaded firmware the EMBArk web page displays error messages.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh
Upload any FW file
Do a scan
Try to delete the uplaoded FW
See error

Expected behavior
Deletion should be working.

Screenshots

Desktop (please complete the following information):

OS: [Kali Linux 2022.01]

daphne could not be found within PATH or Pipfile's [scripts]:

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh

OS: [Kali Linux 2023.01]

[ JOB] Starting migrations - log to embark/logs/migration.log
Traceback (most recent call last):
File "/var/www/embark/./manage.py", line 14, in main
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/var/www/embark/./manage.py", line 26, in
main()
File "/var/www/embark/./manage.py", line 17, in main
raise ImportError(
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
Traceback (most recent call last):
File "/var/www/embark/./manage.py", line 14, in main
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

[ JOB] Collecting static files
Traceback (most recent call last):
File "/var/www/embark/./manage.py", line 14, in main
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

[ JOB] Starting runapscheduler
Traceback (most recent call last):
File "/var/www/embark/./manage.py", line 14, in main
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

[ JOB] Starting Apache
Traceback (most recent call last):
File "/var/www/embark/./manage.py", line 14, in main
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'

The above exception was the direct cause of the following exception:

[ JOB] Starting daphne(ASGI) - log to /embark/logs/daphne.log
Error: the command daphne could not be found within PATH or Pipfile's [scripts].

=============================================================

EMBA logs are under /var/www/emba_logs/

Server started on http://embark.local

log path configuration in EMBArk

Hi,
I would like to scan large bin files with EMBArk and the default 80GB disk space of kali is not enough to execute the test. First I just added a new partition but EMBArk was installed on the default /dev/sda1 partition and EMBArk does not support log parameter, it used the installation partition.
Now I have a separate /dev/sda3 and I installed EMBARk to this folder but in the future it would be great to install EMBArk to the default partition and if I would use the same -l option as EMBA offers I could put the logs and extracted items to a separate disk for huge binary scans.
If possible please look into the possibility of this.

Minor findings in installation, startup and usage of EMBArk

Describe the bug
During the installation the following errors came:

Starting EMBArk displays these issues:

Uploading a firmware file works fine but the path looks like this:

The progress bar did not go to 100% but the san has been finished:

I was able to reproduce this issue more times. I know the scan is finished when the fan slows down in the computer but the progress bar does not go to 100%.

I entered these parameters for the scan:

but the report contains these:

It would be very useful to show real version numbers and proper data in the report, because if I have more scans difficult to check the real versions if the data is incomplete.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh
Use the firmware available here:
https://cyberforat.squat.net/openwrt/binary_images/linksys_official/WRT54GS_3.17.4_US_code.zip
See error

Expected behavior
Please fix the errors if posible.

Screenshots
Attached above.

Desktop (please complete the following information):

OS: Ubuntu 22.04.1 LTS Desktop

Reporter-logreader problem

Describe the bug
EMBA returns successfully but status reporting doesn't work correctly
and boundedexec seems to be stuck

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh
Test some fw
See error

Expected behavior
Exceedingly long emba process seems to crash something?

Screenshots

Desktop (please complete the following information):

OS: ubu22:latest

Wantes to access dashboard via ip from remote host within same network

Hello, I deployed EMBA first and realized that I need a dashboard also so I deployed embark and this is in the esx server running a VM with Ubuntu Server 22.04.

I added
10.x.x.x embark.local
embark.local 10.x.x.x
in /etc/hosts

edited /var/www/httpd80/httpd.conf and added ip under redirect permanent but still I couldn’t access the dashboard using http://10.x.x.x:80 within being in the same network.

Please advise.

Different lengths of labels on X axis

Describe the bug
The graph uses non-uniform long captions, so the size may be smaller due to wrong titles.

To Reproduce
Steps to reproduce the behavior:

Do many scans in EMBArk for different FWs.
Check the OS distribution's graph
See error

Expected behavior
Uniform long labels.

Screenshots

Desktop (please complete the following information):

OS: Ubuntu Server 22.04.1

CVE db issue on Ubuntu Server

Describe the bug
I installed EMBArk in docker mode and I did not find the CVE download section at the end of the installation. The installation output was redirected into a file, which is uploaded to this report. When I started the EMBArk server I got some errors about cve-search is not OK.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (docker-mode)
Start EMBArk: sudo ./run-server.sh
See error

Expected behavior
I know the issue 187 workarounds but I tried 3 times the instalation on a clean Ubuntu Server and I got the same errors all the time.
I would like help to get the installer to handle the database download.

Screenshots

Desktop (please complete the following information):

OS: Ubuntu Server 22.04.1

Additional context
Full installation log
install.txt

Kali Linux 2022.2 released

Kali Linux 2022.2 released here: https://www.kali.org/blog/kali-linux-2022-2-release/
We need to test EMBArk on it

Import device-toggle

Describe the bug
Import-dashboard device-toggle not working

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh
See problem when trying to import

Expected behavior
Toggle doesn't work

Screenshots

Desktop (please complete the following information):

OS: Ubu 22

Analysis failed in EMBArk only on dashboard

Describe the bug
I executed an EMBArk and an emba scan for the same kkeps.bin file.
The problem is that the analysis failed in EMBArk.

I used the:
$ sudo ./emba -l ~/Kankun -f ~/kkeps.bin -p ./scan-profiles/default-scan.emba -X 1.0 -Y Kankun -Z Smartplug -N kkeps

command and it was fine, I got the report at the end.

In EMBArk the report also available, but I can get it from the file system only, and the Dashboard does not show any test results and the report page does not allow downloading the outcome.

Please see the logs from emba and from EMBArk below.

To Reproduce
Run a cwe scan for kkeps.bin in EMBArk.

Steps to reproduce the behavior:

EMBArk installation default mode
Start EMBArk: sudo ./run-server.sh -a 192.168.0.X
Use the firmware available here: see below.
Analysis Failed.

Expected behavior
EMBArk should allow to download the created logs.

Screenshots
Like above.

Desktop (please complete the following information):

OS: Kali Linux 2022.4 for emba and Ubuntu Server 22.04.1 for EMBArk

Additional context
Logs from emba:
emba.log

Logs from EMBArk:
emba_run.log

FW file:
kkeps.zip

Can't acccess to EMBArk (Forbidden HTTP response)

Describe the bug
Can't access to the web server. Forbidden HTTP response.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh

Expected behavior
Access to EMBArk

Screenshots

Desktop (please complete the following information):

OS: Ubuntu Server 22.04 LTS

Additional context
Maybe the reason why of the issue

Unable to connect to embark.local

Describe the bug
I start the run-server.sh and it drops these errors:

Loading .env environment variables...
Traceback (most recent call last):
File "/home/seclab/embark/.venv/bin/daphne", line 8, in
sys.exit(CommandLineInterface.entrypoint())
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/daphne/cli.py", line 170, in entrypoint
cls().run(sys.argv[1:])
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/daphne/cli.py", line 232, in run
application = import_by_path(args.application)
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/daphne/utils.py", line 12, in import_by_path
target = importlib.import_module(module_path)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/seclab/embark/www/embark/./embark/asgi.py", line 20, in
asgi_application = get_asgi_application()
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/django/core/asgi.py", line 12, in get_asgi_application
django.setup(set_prefix=False)
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/home/seclab/embark/.venv/lib/python3.9/site-packages/django/apps/config.py", line 228, in create
import_module(entry)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mod_wsgi'

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh
See error

Expected behavior
I can not get http://embark.local in the web browser.
Message is this:

"Unable to connect
Firefox can’t establish a connection to the server at embark.local."

Desktop (please complete the following information):

OS: [Kali Linux 2022.01]

Additional context
I reinstalled the EMBArk but got the same error. During the install I have these errors:

Installing dependencies from Pipfile.lock (df2d80)...
An error occurred while installing mod-wsgi-httpd==2.4.48.1 --hash=sha256:4ab310a26dcad14246a66b0440c61a2026fedb9e9face8f2989955881907e1f2! Will try again.
An error occurred while installing mod-wsgi-standalone==4.9.0 --hash=sha256:71a5bb625c8810325cd16b8c7ddab69041ba3a1f52f66e4d87aa8e76d582c255! Will try again.
An error occurred while installing mysqlclient==2.1.0 --hash=sha256:02c8826e6add9b20f4cb12dcf016485f7b1d6e30356a1204d05431867a1b3947 --hash=sha256:973235686f1b720536d417bf0a0d39b4ab3d5086b2b6ad5e6752393428c02b12 --hash=sha256:2c8410f54492a3d2488a6a53e2d85b7e016751a1e7d116e7aea9c763f59f5e8c --hash=sha256:e6279263d5a9feca3e0edbc2b2a52c057375bf301d47da2089c075ff76331d14 --hash=sha256:b62d23c11c516cedb887377c8807628c1c65d57593b57853186a6ee18b0c6a5b! Will try again.

Cannot access HTTP from external sources

I use vps ubuntu22.04 to install ebmark,Unable to access after installation is complete

Pipfile.lock install faild

Installation of the firmware scanning environment EMBArk
Reading package lists...
Building dependency tree...
Reading state information...
build-essential is already the newest version (12.9).
default-libmysqlclient-dev is already the newest version (1.1.0).
The following packages were automatically installed and are no longer required:
g++-9 libstdc++-9-dev
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 461 not upgraded.
Requirement already satisfied: pipenv in /usr/lib/python3/dist-packages (2022.12.19)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
www-embark

Install embark daemon

Creating SSL Cert

Certs already generated, skipping

Install hostnames for local dns-resolve

hostname already in use!
Installing dependencies from Pipfile.lock (bcd04c)...
An error occurred while installing apscheduler==3.10.0 ; python_version >= '3.6' --hash=sha256:575299f20073c60a2cc9d4fa5906024cdde33c5c0ce6087c4e3c14be3b50fdd4 --hash=sha256:a49fc23269218416f0e41890eea7a75ed6b284f10630dcfe866ab659621a3696! Will try again.
An error occurred while installing asgiref==3.6.0 ; python_version >= '3.7' --hash=sha256:9567dfe7bd8d3c8c892227827c41cce860b368104c3431da67a0c5a65a949506 --hash=sha256:71e68008da809b957b7ee4b43dbccff33d1b23519fb8344e33f049897077afac! Will try again.
An error occurred while installing async-timeout==4.0.2 ; python_version >= '3.6' --hash=sha256:8ca1e4fcf50d07413d66d1a5e416e42cfdf5851c981d679a09851a6853383b3c --hash=sha256:2163e1640ddb52b7a8c80d0a67a08587e5d245cc9c553a74a847056bc2976b15! Will try again.
An error occurred while installing attrs==22.2.0 ; python_version >= '3.6' --hash=sha256:29e95c7f6778868dbd49170f98f8818f78f3dc5e0e37c0b1f474e3561b240836 --hash=sha256:c9227bfc2f01993c03f68db37d1d15c9690188323c067c641f1a35ca58185f99! Will try again.
An error occurred while installing autobahn==23.1.2 ; python_version >= '3.7' --hash=sha256:c5ef8ca7422015a1af774a883b8aef73d4954c9fcd182c9b5244e08e973f7c3a! Will try again.
An error occurred while installing automat==22.10.0 --hash=sha256:e56beb84edad19dcc11d30e8d9b895f75deeb5ef5e96b84a467066b3b84bb04e --hash=sha256:c3164f8742b9dc440f3682482d32aaff7bb53f71740dd018533f9de286b64180! Will try again.
An error occurred while installing certifi==2022.12.7 ; python_version >= '3.6' --hash=sha256:35824b4c3a97115964b408844d64aa14db1cc518f6562e8d7261699d1350a9e3 --hash=sha256:4ad3232f5e926d6718ec31cfc1fcadfde020920e278684144551c91769c7bc18! Will try again.
An error occurred while installing cffi==1.15.1 --hash=sha256:04ed324bda3cda42b9b695d51bb7d54b680b9719cfab04227cdd1e04e5de3104 --hash=sha256:198caafb44239b60e252492445da556afafc7d1e3ab7a1fb3f0584ef6d742375 --hash=sha256:2012c72d854c2d03e45d06ae57f40d78e5770d252f195b93f581acf3ba44496e --hash=sha256:3799aecf2e17cf585d977b780ce79ff0dc9b78d799fc694221ce814c2c19db83 --hash=sha256:1e74c6b51a9ed6589199c787bf5f9875612ca4a8a0785fb2d4a84429badaf22a --hash=sha256:3bcde07039e586f91b45c88f8583ea7cf7a0770df3a1649627bf598332cb6984 --hash=sha256:40f4774f5a9d4f5e344f31a32b5096977b5d48560c5592e2f3d2c4374bd543ee --hash=sha256:3d08afd128ddaa624a48cf2b859afef385b720bb4b43df214f85616922e6a5ac --hash=sha256:a66d3508133af6e8548451b25058d5812812ec3798c886bf38ed24a98216fab2 --hash=sha256:87c450779d0914f2861b8526e035c5e6da0a3199d8f1add1a665e1cbc6fc6d02 --hash=sha256:a0f100c8912c114ff53e1202d0078b425bee3649ae34d7b070e9697f93c5d52d --hash=sha256:e263d77ee3dd201c3a142934a086a4450861778baaeeb45db4591ef65550b0a6 --hash=sha256:3548db281cd7d2561c9ad9984681c95f7b0e38881201e157833a2342c30d5e8c --hash=sha256:8102eaf27e1e448db915d08afa8b41d6c7ca7a04b7d73af6514df10a3e74bd82 --hash=sha256:a591fe9e525846e4d154205572a029f653ada1a78b93697f3b5a8f1f2bc055b9 --hash=sha256:e00b098126fd45523dd056d2efba6c

embark default installation failed "No module named 'distro'"

Describe the bug

embark default installation fails with "No module named 'distro'" message

To Reproduce

Using kali as base clone the embark repository and run sudo ./installer.sh -d

Expected behavior

embark builds and the server can be started

Screenshots

** Logs **

install_failed.txt

Cannot register user. No access to webapp

Problem accessing EMBark. Cannot register user

Curl user data is invalid OR Web app form "Something went wrong when signing up the user."

Authentication
Before accessing EMBArk you need to register yourself with username and password:

Option 1:
curl -XPOST 'http://0.0.0.0:80/signup' -d '{"email": "[email protected]", "password": "test", "confirm_password": "test"}'

└──╼ $curl -XPOST 'http://127.0.0.1:80/signup' -d '{"email": "[email protected]", "password": "test", "confirm_password": "test"}'

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8"/>
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"/>

    <link rel="icon" type="image/png" href="/static/content/images/favicon.png"/>

    <title>EMBArk register</title>

    <!-- jQuery-confirm style sheet -->
    <link rel="stylesheet" href="/static/external/css/confirm.css"/>

    <!-- Bootstrap and jquery-confirm style sheet -->
    <link rel="stylesheet" href="/static/external/css/bootstrap.css"/>

    <!--DataTables style sheets-->
    <link rel="stylesheet" type="text/css" href="/static/external/css/datatable.css"/>

    <!-- Style sheets-->
    <link rel="stylesheet" type="text/css" href="/static/content/css/globalStyle.css"/>
    <link rel="stylesheet" type="text/css" href="/static/content/css/login.css"/>

    <!-- jQuery script -->
    <script type="text/javascript" src="/static/external/scripts/jquery.js"></script>

    <!-- jQuery confirm script -->
    <script type="text/javascript" src="/static/external/scripts/confirm.js"></script>

    <!-- Bootstrap script -->
    <script type="text/javascript" src="/static/external/scripts/bootstrap.js"></script>

    <!--DataTables script-->
    <script type="text/javascript" src="/static/external/scripts/datatable.js"></script>

    <!-- Charts script -->
    <script type="text/javascript" src="/static/external/scripts/charts.js"></script>

    <!-- local Javascript files-->
    <script type="text/javascript" src="/static/scripts/main.js"></script>
    <script type="text/javascript" src="/static/scripts/alertBox.js"></script>


</head>
<body>
    <div class="container-fluid">

        <!--Main container-->
        <div class="main">

    <div class="login-form-container">

        <div class="alert alert-danger alert-dismissible fade show" role="alert">
            <medium>User data is invalid.</medium>
        </div>

        <div id="embarkLogo"><img src="/static/content/images/embark_logo.svg" alt="EMBArk logo graphic" height="auto" width="auto"/></div>
        <div class="login">
            <form action="/signup" class="login-form" method="POST" novalidate>
            <h2 class="title">Register</h2>
            <div class="input-field" data-error="Username is required">
                <svg viewBox="0 -2 25 25" xmlns="http://www.w3.org/2000/svg"><path d="m7.5.5c1.65685425 0 3 1.34314575 3 3v2c0 1.65685425-1.34314575 3-3 3s-3-1.34314575-3-3v-2c0-1.65685425 1.34314575-3 3-3zm7 14v-.7281753c0-3.1864098-3.6862915-5.2718247-7-5.2718247s-7 2.0854149-7 5.2718247v.7281753c0 .5522847.44771525 1 1 1h12c.5522847 0 1-.4477153 1-1z" fill="none" stroke="#000" stroke-linecap="round" stroke-linejoin="round" transform="translate(3 2)"/></svg>
                <input type="text" placeholder="Username" name="username" required/>
            </div>
            <div class="input-field">
                <svg viewBox="0 -2 25 25" xmlns="http://www.w3.org/2000/svg"><g fill="none" fill-rule="evenodd" transform="translate(4 1)"><path d="m2.5 8.5-.00586729-1.99475098c-.00728549-4.00349935 1.32800361-6.00524902 4.00586729-6.00524902s4.0112203 2.00174967 4.0000699 6.00524902v1.99475098m-8.0000699 0h8.0225317c1.0543618 0 1.9181652.81587779 1.9945143 1.8507377l.0054778.1548972-.0169048 6c-.0031058 1.1023652-.8976224 1.9943651-1.999992 1.9943651h-8.005627c-1.1045695 0-2-.8954305-2-2v-6c0-1.1045695.8954305-2 2-2z" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><circle cx="6.5" cy="13.5" fill="#000" r="1.5"/></g></svg>
                <input type="password" placeholder="Password" name="password" required/>
            </div>
            <div class="input-field">
                <svg viewBox="0 -2 25 25" xmlns="http://www.w3.org/2000/svg"><g fill="none" fill-rule="evenodd" transform="translate(4 1)"><path d="m2.5 8.5-.00586729-1.99475098c-.00728549-4.00349935 1.32800361-6.00524902 4.00586729-6.00524902s4.0112203 2.00174967 4.0000699 6.00524902v1.99475098m-8.0000699 0h8.0225317c1.0543618 0 1.9181652.81587779 1.9945143 1.8507377l.0054778.1548972-.0169048 6c-.0031058 1.1023652-.8976224 1.9943651-1.999992 1.9943651h-8.005627c-1.1045695 0-2-.8954305-2-2v-6c0-1.1045695.8954305-2 2-2z" stroke="#000" stroke-linecap="round" stroke-linejoin="round"/><circle cx="6.5" cy="13.5" fill="#000" r="1.5"/></g></svg>
                <input type="password" placeholder="Confirm password" name="confirm_password" required/>
            </div>
            <input id="loginButton" type="submit" class="solid btn-login" value="Register" />
            </form>
        </div>
        <div id="login_footer">
            <a href="/">
                <input class="solid btn-login" type="submit" value="Back" />
            </a>
        </div>
    </div>

        </div>
    </div>

</body>
</html>

Initial implementation improvement list

Original issue: amosproj/amos2021ss01-emba-service#165

implement cancel button behaviour
add request for reloading service dashboard
implement clean up in logreader especially the processmap

hashid

important step in making EMBArk secure and robust.
switch from normal integer-id to uuid/hashid primary-key

HTML toggles for false positives

This issue was originally reported in the EMBA area: e-m-b-a/emba#193

This is a tall order but would be nice for the roadmap

In most cases. the discoveries for the CVEs don't actually affect the product. For example, if I'm running a kernel version that has 200 CVE's and 7 exploits. When I look at those findings I notice the CVE's are just a raw version analysis but if you dig down into the CVE it can say stuff like "If IPV6 is enabled" "IF the following flag is enabled in x config". IT would be nice to have the ability to go into the HTML report and maybe toggle stuff off that you know is a false positive.

Kina like this project lets you do https://github.com/Guezone/SECMON.

The toggling could let you generate an XML or something that logs the CVE's that you could apply to your next scan --fpxml

Pip-enviroment not found!

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh

OS: [Kali Linux 2023.1]

Smaller graph on Device Tracker page

Is your feature request related to a problem? Please describe.
The Device Tracker shows a graph of the selected item but the size is too large in my opinion. Maybe half of the size would be enough and the background picture quality is low on this page. I know it’s a minor issue, and maybe it is only my opinion, but I think the readability would be better in smaller size.

Describe the solution you'd like
I think on an average screen resolution 1920x1200 the graph should be fit on the page with all categories on the top, below and sides to see the whole status of the actual FW charactersitics.

Installation Errors on Fresh Ubuntu 22.04.1 Install and Unable to Connect to embark.local

Describe the bug
I have a fresh Minimal install of Ubuntu 22.04.1 (fully updated) on bare metal just for Emba and Embark

Installing Emba seems to have gone flawlessly but Installing Embark gave some unexpected errors (first two images below) and when starting the server via
sudo ./run-server.sh it gives a "Failed setup mysql and redis docker images" error (3rd image below). When attempting to go to embark.local it just says a browser Unable to Connect message.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
See errors
Start EMBArk: sudo ./run-server.sh
See error

Expected behavior
I can not get http://embark.local/ in the web browser.
Message is this:
"Unable to connect
Firefox can’t establish a connection to the server at embark.local."

Screenshots

Desktop (please complete the following information):

OS: Ubuntu 22.04.1 (Jammy) Fully Patched / Updated

Additional context
I have re-run the sudo ./installer.sh -d command after rebooting and get same issues again.

main-dashboard empty and bugged

Describe the bug
script has major flaws and problems.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh
run any fw
main dash empty
See error

Expected behavior
main dashboard is expected to display clear and concise information

Screenshots

Desktop (please complete the following information):

OS: Ubu22.04

Additional context
😠

EMBArk login fails

Hi,

I am using a Kali 2021.04 linuxin VMware Workstation.
I am doing a partly manual installation of emba + EMBArk like this:

change shell to /bin/bash
docker installation:
$ sudo apt install docker-ce -y

$ git clone https://github.com/e-m-b-a/embark.git
$ cd embark
$ git clone https://github.com/e-m-b-a/emba.git
$ git clone https://github.com/cve-search/cve-search.git

Install system requirements:
$ cd cve-search
$ sudo xargs apt-get install -y < requirements.system
$ sudo pip3 install -r requirements.txt

MongoDB installation
$ sudo apt install -y mongodb-org

and I install EMBArk with the default -d mode. The install looks fine, I can start the server with:
$ sudo ./run-server.sh
and I can register a user in the browser but after login I get 404 error.

If I start with developer mode:
sudo ./dev-tools/debug-server-start.sh
I can log in.

I checked also the -F installation which is also looks fine, but if I start the server with the default command:
$ sudo ./run-server.sh

I get this error:
Finished setup mysql and redis docker images
mkdir: cannot create directory ‘/app/www/logs’: No such file or directory
mkdir: cannot create directory ‘/app/www/conf’: No such file or directory

[ JOB] Redis logs are copied to ./embark/logs/redis_dev.log

[ JOB] DB logs are copied to ./embark/logs/mysql_dev.log
./run-server.sh: line 98: /app/www/logs/redis.log: No such file or directory
./run-server.sh: line 100: /app/www/logs/mysql.log: No such file or directory
cp: cannot create directory '/app/www/embark/': No such file or directory
./run-server.sh: line 113: /app/www/conf/embark.conf: No such file or directory
./run-server.sh: line 116: cd: /app/www/embark/: No such file or directory

The output of docker-compose ps:

Name Command State Ports

embark_db docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp,:::3306->3306/tcp, 33060/tcp
embark_redis docker-entrypoint.sh --por ... Up 6379/tcp, 0.0.0.0:7777->7777/tcp,:::7777->7777/tcp

I can use only the
$ sudo ./dev-tools/debug-server-start.sh

mode even if I install -d or with -F but the normal way does not work.
What could be the problem?

If I want to open a report after a full emba test in EMBArk I get this message:

TemplateDoesNotExist at /emba_logs/1/html-report/index.html

/app/emba/emba_logs/emba_logs/1/html-report/index.html

Request Method: GET
Request URL: http://127.0.0.1:8000/emba_logs/1/html-report/index.html
Django Version: 4.0.1
Exception Type: TemplateDoesNotExist
Exception Value:

/app/emba/emba_logs/emba_logs/1/html-report/index.html

Exception Location: /home/kali/embark/.venv/lib/python3.9/site-packages/django/template/loader.py, line 19, in get_template
Python Executable: /home/kali/embark/.venv/bin/python
Python Version: 3.9.9
Python Path:

['/home/kali/embark/embark',
'/home/kali/embark',
'/home/kali/embark/embark',
'/usr/lib/python39.zip',
'/usr/lib/python3.9',
'/usr/lib/python3.9/lib-dynload',
'/home/kali/embark/.venv/lib/python3.9/site-packages']

Server time: Sun, 30 Jan 2022 10:14:42 +0000
Template-loader postmortem

Django tried loading these templates, in this order:

Using engine django:

django.template.loaders.filesystem.Loader: /app/emba/emba_logs/emba_logs/1/html-report/index.html (Source does not exist)

But other pages are working and I am able to download the logs in HTML with "Download Logs" button.

Regards,
Torabi

EMBArk fast test

Hello Benedikt,

I have seen the new release today and I installed a new Kali 2022.3 VM afternon with the actual EMBArk.
Installation was fine I think.

Starting the server dropped and error but the service was reachable after this:

I did a short scan but the progress bar stopped at 91%:

The scan was over for sure.

After this I checked the details of the scan and the page only displayed the enthropy graph:

I though it was because of the wrong FW file and I wanted to start the 2nd scan but I got an error after I wanted to start the scan:

I logged out and stopped EMBArk and started again. I got the first No such service: EMBA error like on the first screenshot and I couldn't start a new scan because the last error came up again.

I tried again the same with a brand new installation.
A stating error in the terminal came again. I was able to start the 2nd scan. Please ignore the empty page issue of the details page, because another FW scan displayed the contents fine.
I am not sure the progress bar works perfect or not but I am going to try and check it some more times. Maybe the fix is still in progress for this.

I also checked the firmware delete page and it was blank:

Thank you!
torabi

Installation logfile and OVF export questions

Describe the bug
Hello Benedikt,

1st question:
Could you please help me where I can find the log file of EMBArk installation on Ubuntu server?

2nd question:
I installed EMBArk on Ubuntu Server 22.04.1 and works fine but I exported the system to an OVF file and I imported in ESXi 8.0. The scan is working also and I get sucsess message on ESXi but there is no report and download log for the scan.
Could you help which log would be valuable for this problem?

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode)
Start EMBArk: sudo ./run-server.sh -a
Use the firmware available here: Sorry don't remember but the scan was OK.
After the scan there is no report in EMBArk in case of host ESXi 8.0 environment.

Desktop (please complete the following information):

OS: Ubuntu Server 22.04.1

Name conventions

Describe the bug
I created a "D-Link" vendor in EMBArk but the test did not start, because the name contains hyphen, which is blocked by EMBA.
I tried to start the same test without selecting the existing vendor but the test also did not start. I created again a new scan with DLink vendor and it worked fine:

To Reproduce
Steps to reproduce the behavior:

EMBArk installation default-mode
Start EMBArk: sudo ./run-server.sh -a
Use the firmware available here: anything
Create a vendor or device with a non supported character and assign it to the FW file and try to start the scan.
There is no error, the test stops after a few seconds.

Expected behavior
EMBArk should warn the user and only the supported characters would be accepted on the page.

Screenshots
See above.

Desktop (please complete the following information):

OS: Ubuntu Server 22.04.1

s109 page is not reachable

Describe the bug
I used a custom wordlist file in /embark/emba/config/jtr_wordlist.txt, which contained the good password for the scanned FW.
I executed the same default-profile scan in emba and in EMBArk, and the s109 modul sub page dropped UnicodeDecodeError in the browser, see screenshots.

To Reproduce
Steps to reproduce the behavior:

EMBArk installation default mode
Start EMBArk: sudo ./run-server.sh -a 192.168.x.x
Use the firmware available here: kkeps.bin
Open "Cracking identified password hashes" menu
See error

Expected behavior
Page should be displayed in EMBArk.

Screenshots
emba:

EMBArk:

Desktop (please complete the following information):

OS: Ubuntu Server 22.04.1

Additional context
I used this wordlist files with custom modification:
https://github.com/berzerk0/Probable-Wordlists/tree/master/Real-Passwords

Top109Million-probable-v2.txt = jtr_wordlist.txt

File-state-bug

Describe the bug
Files are not present post reinstallations.
(report-dashboard)

To Reproduce
Steps to reproduce the behavior:

EMBArk installation (default mode/docker-mode)
Start EMBArk: sudo ./run-server.sh
use
reinstall
See error when trying to open old analysis

Expected behavior
Error message or analysis buttons greyed out

Screenshots

Desktop (please complete the following information):

OS:Ubu 22

Pip-enviroment not found!

Routersploit CVE database - ok
Metasploit CVE database - ok
checksec script - ok
sshdcc script - ok
sudo-parser script - ok
shellcheck script - ok
fdtdump - ok
linux-exploit-suggester.sh script - ok
objdump disassembler - ok
radare2 - ok
bandit - python vulnerability scanner - ok
qemu-[ARCH]-static - ok
yara - ok
cyclonedx - ok
vmlinux-to-elf - ok
STACS hash detection - ok
Qemu system emulator ARM - ok
Qemu system emulator ARM64 - ok
Qemu system emulator MIPS - ok
Qemu system emulator MIPSel - ok
Qemu system emulator MIPS64 - ok
Qemu system emulator MIPS64el - ok
Qemu system emulator NIOS2 - ok
Qemu system emulator x86 - ok
console.* - ok
busybox.* - ok
libnvram.* - ok
libnvram_ioctl.* - ok
vmlinux.mips* - ok
zImage.armel - ok
fixImage.sh - ok
preInit.sh - ok
inferFile.sh - ok
inferService.sh - ok
Routersploit installation - ok
Arachni web scanner installation - ok
TestSSL.sh installation - ok
Nikto web server analyzer - ok
Cutycapt screenshot tool - ok
snmp-check tool - ok
Nmap portscanner - ok
hping3 - ok
ping - ok
Metasploit framework - ok
Running Qemu network service - ok
Running Qemu telnet service - ok
cwe-checker environment - ok

Pip-enviroment not found!
root@ccehludrkintgi:~/embark#

each firmware scan takes too much time,nearly 2 hour

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Uncontrolled data used in path expression

uploader/views.py
line 366

emba-input

Describe the bug
whitelists don't align adequately

To Reproduce
try running analysis with device-name containing "-"
5. See error

Expected behavior
solution 1: change emba validation
solution 2: change EMBArks validation for inputs

Screenshots

Desktop (please complete the following information):

OS: Ubu22

EMBArk not working due to emba not installing correctly(using latest commit)

Hey 👋

Describe the bug
I'm currently testing out EMBArk and I had couple issue setting it up.
It seems that currently freetz-ng and fitmg fail to install.

This might be related to the fact that the current emba is not an official release but instead the latest master development branch(107d304).

To Reproduce
Steps to reproduce the behavior:

Clone the current EMBArk repository
EMBArk installation (default mode)
Fails

Expected behavior
Just EMBArk that finish the installation without failing

Desktop (please complete the following information):
I'm currently using an Ubuntu 22.04 LTS server as recommended.

Screenshots
Here is a screenshot of the issue:

Server start does not work

Describe the bug
I did a clean installation on Kali using the default "sudo installer.sh -d" and it went fine, but I could not start the server:

To Reproduce
Steps to reproduce the behavior:

EMBArk installation
$ git clone https://github.com/e-m-b-a/embark.git
$ cd embark
$ sudo ./installer.sh -d
Start EMBArk: sudo ./run-server.sh
See error

Expected behavior
It used to work the same way a few weeks before. May I install MongoDB manually? I didn't find any error during the installation.

Screenshots
Added above.

Desktop (please complete the following information):

OS: [Kali Linux 2022.01] & sudo apt full-upgrade -y

e-m-b-a / embark Goto Github PK

embark's Introduction

EMBArk - The firmware security scanning environment

About

Important

Quick-Start

Automated setup on Ubuntu 22.04 LTS

Start Server

Upgrading

Developer

Get involved

Acknowledgement

embark's People

Contributors

Stargazers

Watchers

Forkers

embark's Issues

EMBA-version requires update

Name Command State Ports

Recommend Projects

Recommend Topics

Recommend Org

Jobs