e-m-b-a / emba Goto Github PK
View Code? Open in Web Editor NEWEMBA - The firmware security analyzer
Home Page: https://www.securefirmware.de
License: GNU General Public License v3.0
EMBA - The firmware security analyzer
Home Page: https://www.securefirmware.de
License: GNU General Public License v3.0
See #70
Let's install fact-extractor locally and in our emba docker image. With this we do not need the docker image of fact-extractor anymore and fix our security issues.
Same issue like #38 - emba isn't able to extract firmware with Fact_extractor if started with -D
.
Kali Linux 2021.1 is out: https://www.kali.org/downloads/
We need to test emba and all the deps on the new release
With the pull request from @Anemosx we now have full Docker support in emba.
As discussed here #66 (comment) this also includes new security issues. Nevertheless we see a massive improvement compared to running emba on your host.
Let's use this Issue to discuss further steps.
See here https://www.kali.org/blog/kali-linux-2021-3-release/
Testcases:
This is a tall order but would be nice for the roadmap
In most cases. the discoveries for the CVEs don't actually affect the product. For example, if I'm running a kernel version that has 200 CVE's and 7 exploits. When I look at those findings I notice the CVE's are just a raw version analysis but if you dig down into the CVE it can say stuff like "If IPV6 is enabled" "IF the following flag is enabled in x config". IT would be nice to have the ability to go into the HTML report and maybe toggle stuff off that you know is a false positive.
Kina like this project lets you do https://github.com/Guezone/SECMON.
The toggling could let you generate an XML or something that logs the CVE's that you could apply to your next scan --fpxml
I got docker-compose name error on updated Ubuntu 20.04 while installing.
According to the solution I found on stackoverflow, adding this to the install script will fix the problem:
sudo curl -L https://github.com/docker/compose/releases/download/1.28.5/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Referance:
https://stackoverflow.com/questions/58155523/unable-to-give-network-name-in-docker-compose
I'm running in a fresh Ubuntu VM and cloned EMBA from git then ran the installer and yes, it seems to have failed to install the net-tools.
I could (untested!) imagine that the way you handle parameters with a dash (-) in it, that it somehow breaks because the dash is interpreted as a command line parameter switch. But that's just wild speculation.
Originally posted by @floyd-fuh in #118 (comment)
New Kali Linux is available. We need to test everything on it
Describe the bug
Hello, while running the install (in embabox) a fresh install currently fails.
default: [+] IP60_fact_extractor
default: =================================================================
default: fact-extractor
default: Description: Wraps FACT unpack plugins into standalone utility. Should be able to extract most of the common container formats. (EMBA fork)
default: Download-Size: 34 MB
default: fact_extractor will be downloaded.
default:
default: FACT-extractor will be downloaded and installed!
default: Reading package lists...
default: Building dependency tree...
default:
default: Reading state information...
default: The following additional packages will be installed:
default: libcurl4 libldap-2.5-0
default: The following NEW packages will be installed:
default: libldap-2.5-0
default: The following packages will be upgraded:
default: curl libcurl4
default: 2 upgraded, 1 newly installed, 0 to remove and 948 not upgraded.
default: Need to get 869 kB of archives.
default: After this operation, 581 kB of additional disk space will be used.
default: Do you want to continue? [Y/n]
default: Abort.
default: ------------------------------------------------------------------------------------------------
default: Error detected - status code 1
default: Command: echo -e "Command: $ORANGE$BASH_COMMAND$NC"
default: Location: ./installer/IP60_fact_extractor.sh, line 40
default: Stack Trace:
default: [1] IP60_fact_extractor(): ./installer/IP60_fact_extractor.sh, line 40 -> IP60_fact_extractor
default: [2] main(): ./installer.sh, line 168 -> main -F
default:
default: Important: Consider filling out a bug report at https://github.com/e-m-b-a/emba/issues
default: ------------------------------------------------------------------------------------------------
It looks like the curl install needs at least a -y flag to install without interaction?
https://github.com/e-m-b-a/emba/blob/master/installer/IP60_fact_extractor.sh#L40
Describe the bug
In case of PROXY the default installation of emba stops at I05_emba_docker_image download:
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The image download should be working fine.
Screenshots
Added above.
Desktop (please complete the following information):
@firmianay found some issues in the installation script that need to be checked.
Previous discussion: #77
From the logfile, I conclude that certain files in /etc where not found by the tool. Am I correct?
These files are present in the tested firmware image!
[+] Check users, groups and authentication
=================================================================
==> Users with UID zero (0)
-----------------------------------------------------------------
[*] Searching accounts with UID 0
[-] /etc/passwd not available
==> Non-unique accounts
-----------------------------------------------------------------
[*] Searching non-unique accounts
[-] /etc/passwd not available
==> Unique group IDs
-----------------------------------------------------------------
[*] Searching non-unique group ID's
[-] /etc/group not available
==> Unique group name
-----------------------------------------------------------------
[*] Searching non-unique group names
[-] /etc/group not available
==> Query user accounts
-----------------------------------------------------------------
[*] Reading system users
[-] /etc/passwd not available
==> Query NIS and NIS+ authentication support
-----------------------------------------------------------------
[*] Check nsswitch.conf
[-] /etc/nsswitch.conf not available
Is your feature request related to a problem? Please describe.
Since PR #213 EMBA supports reporting templates. These templates can be easily created for the different modules.
The better the templates are, the more useful is this feature.
Describe the solution you'd like
Include templates for every module. These templates should be generic and should describe the module, the tasks and the results.
See an example here:
Some example templates can be found here: https://github.com/e-m-b-a/emba/tree/master/config/report_templates
Looks as the output of the cwe-checker has changed a bit.
@p4cx has completely rewritten the emba html reporter here: #101
With this base we now have great possibilities. This issue is a collection and discussion of some ideas:
Describe the bug
I am getting an error during the installation near metasploit-framework
To Reproduce
Log;
default: tcllib will be newly installed.
default:
default: metasploit-framework
default: ------------------------------------------------------------------------------------------------
default: Error detected - status code 100
default: Command: TOOL_INFO="$(apt show "${1:-}" 2> /dev/null)"
default: Location: ./installer/helpers.sh, line 47
default: Stack Trace:
default: [1] print_tool_info(): ./installer/helpers.sh, line 47 -> print_tool_info metasploit-framework 1
default: [2] I01_default_apps(): ./installer/I01_default_apps.sh, line 33 -> I01_default_apps
default: [3] main(): ./installer.sh, line 156 -> main -F
default:
default: Important: Consider filling out a bug report at https://github.com/e-m-b-a/emba/issues
default:
default: ------------------------------------------------------------------------------------------------
default: bash: line 5: xit: command not found
"xit" command not found -> exit perhaps?
Describe the bug
I am using kali 2021.4 and I have to set up proxy to get internet access. I set the following parameters on my linux:
HTTP_PROXY=A.B.C.D:1234
HTTPS_PROXY=A.B.C.D:1234
NO_PROXY=localhost,127.0.,127.36.
echo "http_proxy=http://${HTTP_PROXY}/" >>/etc/environment &&
echo "https_proxy=http://${HTTPS_PROXY}/" >>/etc/environment &&
echo "no_proxy=${NO_PROXY}" >>/etc/environment
I set the proxy for APT:
touch /etc/apt/apt.conf.d/proxy.conf &&
echo "Acquire::http::Proxy "http://${HTTP_PROXY}";" >>/etc/apt/apt.conf.d/proxy.conf &&
echo "Acquire::https::Proxy "http://${HTTPS_PROXY}";" >>/etc/apt/apt.conf.d/proxy.conf
export:
export http_proxy=${HTTP_PROXY}
export https_proxy=${HTTPS_PROXY}
export no_proxy=${NO_PROXY}
proxy for wget as root:
cp /etc/wgetrc /root/.wgetrc
echo -e "use_proxy = on\nhttp_proxy = http://${HTTP_PROXY}\nhttps_proxy = http://${HTTPS_PROXY}\nftp_proxy = http://${HTTP_PROXY}" >>/root/.wgetrc
proxy for wget as kali:
cp /etc/wgetrc /home/kali/.wgetrc
echo -e "use_proxy = on\nhttp_proxy = http://${HTTP_PROXY}\nhttps_proxy = http://${HTTPS_PROXY}\nftp_proxy = http://${HTTP_PROXY}" >>/home/kali/.wgetrc
and I set proxy for Git:
git config --global http.proxy http://${HTTP_PROXY}
git config --global https.proxy http://${HTTPS_PROXY}
apt upgrade and reboot, and I git lone the emba, then cd ~emba, then:
sudo ./isntaller.sh -d
All the things are fine but after mongoDB installation the setup failes. This is on the screen:
Adding user mongodb' to group
mongodb' ...
Adding user mongodb to group mongodb
Done.
Setting up mongodb-org-shell (4.4.12) ...
Setting up mongodb-database-tools (100.5.2) ...
Setting up mongodb-org-mongos (4.4.12) ...
Setting up mongodb-org-database-tools-extra (4.4.12) ...
Setting up mongodb-org-tools (4.4.12) ...
Setting up mongodb-org (4.4.12) ...
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for kali-menu (2021.4.2) ...
Scanning processes...
Scanning linux images...
Running kernel seems to be up-to-date.
No services need to be restarted.
No containers need to be restarted.
No user sessions are running outdated binaries.
Created symlink /etc/systemd/system/multi-user.target.wants/mongod.service → /lib/systemd/system/mongod.service.
The cve-search database will be downloaded and updated!
Check if the cve-search database is already installed.
cve-search database not ready.
The installer is going to populate the database.
Starting redis-server (via systemctl): redis-server.service.
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_cpe_dictionary.py", line 27, in
from lib.Sources_process import CPEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_json.py", line 22, in
from lib.Sources_process import CVEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_updater.py", line 21, in
from lib.DatabaseSchemaChecker import SchemaChecker
File "/home/kali/emba/external/cve-search/sbin/../lib/DatabaseSchemaChecker.py", line 7, in
from lib.DatabaseHandler import DatabaseHandler
File "/home/kali/emba/external/cve-search/sbin/../lib/DatabaseHandler.py", line 1, in
from lib.ApiRequests import JSONApiRequest
File "/home/kali/emba/external/cve-search/sbin/../lib/ApiRequests.py", line 5, in
from nested_lookup import nested_lookup, nested_update
ModuleNotFoundError: No module named 'nested_lookup'
The cron.daily update script for EMBA is located in config/emba_updater
For automatic updates it should be copied to /etc/cron.daily/
Installation notes:
INFO: The cron.daily update script for EMBA is located in config/emba_updater
INFO: For automatic updates it should be copied to /etc/cron.daily/
INFO: For manual updates just start it via sudo ./config/emba_updater
WARNING: If you plan using the emulator (-E switch) your host and your internal network needs to be protected.
INFO: Do not forget to checkout current development of EMBA at https://github.com/e-m-b-a.
EMBA installation finished
and I get back the prompt.
Could you help what other proxy settings required to get the installation go through the entire process?
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Please verify my proxy settings and if possible please give any advice what to do for population the CVE database by the instalation script as kali user:
$ sudo ./installer.sh -d
Screenshots
I copied the error message above.
Desktop (please complete the following information):
Additional context
Proxy must be set but I am not sure I defined all the needed parameters for the emba installer.
Based on #32 and #28 we have docker support.
Currently our setup does not support cve-search - https://cve-search.github.io/cve-search/
Is your feature request related to a problem? Please describe.
Kali Linux 2022.2 released here: https://www.kali.org/blog/kali-linux-2022-2-release/
We need to test EMBA on it
Testcases:
We have started to include license details in our version identifier configuration here: https://github.com/e-m-b-a/emba/blob/master/config/bin_version_strings.cfg
This is in a very early stage and need your help. If you know the license of some of the software components please include it directly in the configuration and bring up a pull request or comment to this issue with the license and the source for verification.
See here https://www.kali.org/blog/kali-linux-2021-4-release/
Testcases:
See here https://www.kali.org/blog/kali-linux-2022-1-release/
Testcases:
Emba should be able to detect vmdk images e.g. with the file command and then extracts it correctly.
The following documentation works quite nice on our Kali environment: https://unix.stackexchange.com/questions/550569/how-can-i-access-the-files-in-a-vmdk-file
Another documentation: https://serverfault.com/questions/244642/extract-files-from-vmdk
We could mount it and then copy everything to the firmware folder in the log area.
Describe the bug
I found bug in https://github.com/e-m-b-a/emba
=> emba/config/bin_version_strings.cfg/opensslResults when using an existing regular expression
openssl;;bsd;"OpenSSL\ [0-9](\.[0-9]+)+?\ ";"sed -r 's/OpenSSL\ ([0-9](\.[0-9]+)+?).*$/openssl:\1/'"; => OpenSSL 1.1.1 OpenSSL 1.1.1Result when using the modified regular expression
openssl;;bsd;"OpenSSL\ [0-9](\.[0-9]+)+?\ ";"sed -r 's/OpenSSL\ ([0-9](\.[0-9]+)+?)\ /openssl:\1/'"; => openssl 1.1.1It looks like the above regex needs to be modified.
To Reproduce
Information not provided
Expected behavior
Information not provided
Screenshots
Information not provided
Desktop (please complete the following information):
Information not provided
Additional context
I got this bug report via mail. Didn't try to reproduce it.
After running docker-compose I got the following error:
---> 831e1fbfdc4e
Step 4/6 : ADD . /app
---> a575a222b90e
Step 5/6 : RUN yes | ./installer.sh
---> Running in e2ec779affe5
/bin/sh: 1: ./installer.sh: Permission denied
ERROR: Service 'emba' failed to build: The command '/bin/sh -c yes | ./installer.sh' returned a non-zero code: 126
The solution is to chmod +x installer.sh and docker-compose will succeed.
Thank you.
Describe the bug
The emba installer when uses the
pip3 install -r requirements
command this error comes:
To Reproduce
Steps to reproduce the behavior:
Expected behavior
pip3 install should be working fine.
Screenshots
Added above.
Desktop (please complete the following information):
As mentioned here: https://github.com/e-m-b-a/emba/wiki/Installation#cve-search-installation the current installer is not able to install all required dependencies.
Is your feature request related to a problem? Please describe.
I am trying to use emba to generate a report . I am combining the results found here with other tools. Initially, the idea was to go through the CSV files and generate rst or latex report but I am running into small problems. For example
f50_base_aggregator.txt (the text version) is pretty nice in terms of overview but to me it appears the CSV does not contain the same information also while trying to read the CSV it appears that not all the lines in the file have the same amount of fields
architecture_verified;"ARM"
strcpy_bin;"libicui18n.so";"11"
version_details;kernel;4.9.160;CVEs;788;Exploits;197
The main thing I would like in terms of functionality is to be able to query the data but at this point, I don't know if reading the CSV files is the way to go.
Describe the solution you'd like
GIven there is also already a MongoDB database I was thinking perhaps the csv needs to become json and this can be put in the database. Perhaps just converting to json would be enough . I think it would be great to have a shell/ web front-end where you can query the data (on file base perhaps?) and get the combined results of different tests. Does Metasploit not do something similar?
Describe alternatives you've considered
I am currently just including the .txt files in my report (including colors) and emba is really useful to help determine what to perform for a pentest but it would be nice to be able to also use the findings(filter them) in a report.
I Have previously written a plugin to include CVE type data into https://github.com/blacklanternsecurity/writehat but I would currently like to use python to read the data and generate nice graphs/ links etc.
Additional context
I hope we can have a nice discussion on the topic. It might also be possible to rewrite some of the shell scripts to store a bit more context( or store the data into a more computer-friendly format) or perhaps I am missing something
I think we talked about it before but because I ran EMBA again and used these tools afterwards so I thought I let you know.
Is your feature request related to a problem? Please describe.
I think the password hash search and other analysis steps after unpacking in EMBA could be improved.
Describe the solution you'd like
I guess using https://github.com/returntocorp/semgrep is the best choice at the moment because of the huge amount of rules in https://github.com/returntocorp/semgrep-rules
Describe alternatives you've considered
If semgrep doesn't work for some reason there is also my standalone https://github.com/floyd-fuh/crass/blob/master/grep-it.sh script that might be easier to implement. As an fyi, I've been maintaining it for 7 years now and even though there is semgrep I'm not planning to stop. Btw. if you would like to keep your current greps we could integrate them as rules in grep-it.sh if you think something is missing.
Additional context
I know searching for CVEs is probably a quick win in many cases, but when you really want to do proper security research to pwn a device with custom/new approaches/techniques/ideas it is good to have a tool that only points to "fishy" areas (e.g. anything related to crypto as in https://github.com/floyd-fuh/crass/blob/f2da104b073f530fbadeda7578c39a377ebd296b/grep-it.sh#L318 ). In that regards I'm not sure (I really don't know as I've just started to use it) if semgrep allows an approach that is broad enough, whereas in CRASS I'm trying. I would still rather go with semgrep. Or both. If you use semgrep with --config=auto just make sure to tell your users that you are using a third-party (metrics are sent to their servers).
Not totally sure if it's an issue but didn't look right?
└──╼ $sudo ./config/emba_updater
[sudo] password for ovarroadmin:
[*] EMBA update - cve_searchsploit update
./config/emba_updater: 9: cve_searchsploit: not found
[*] EMBA update - cve-search update
Thanks a lot for this great project. I fast forwarded through the wiki but was not able to figure out which embedded Linux operating systems are supported. Probably some distributions are not supported at all. Probably some checks are meant for specific distributions only. There are so many Linux-based embedded OSes like e.g. Android Things, Android Cars, OpenWRT, Yocto Project embedded Linux, Ubuntu Core, uClinux, ... Could you provide more info about distro compatibility?
�[0;35m�[1mThe cve-search database will be downloaded and updated!�[0m
�[0;35mCheck if the cve-search database is already installed.�[0m
�[0;35mcve-search database not ready.�[0m
�[0;35mThe installer is going to populate the database.�[0m
Starting redis-server (via systemctl): redis-server.service.
Traceback (most recent call last):
File "/opt/emba/external/cve-search/./sbin/db_mgmt_cpe_dictionary.py", line 27, in
from lib.Sources_process import CPEDownloads
File "/opt/emba/external/cve-search/sbin/../lib/Sources_process.py", line 17, in
from lib.DatabaseLayer import (
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseLayer.py", line 35, in
mongo_version = db.command("buildinfo")["versionArray"]
File "/usr/local/lib/python3.9/dist-packages/pymongo/database.py", line 757, in command
with self.__client._socket_for_reads(
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1346, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 244, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 202, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 172.36.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 61af3eb03c94e0047bb56ec7, topology_type: Single, servers: [<ServerDescription ('172.36.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.36.0.1:27017: timed out')>]>
Traceback (most recent call last):
File "/opt/emba/external/cve-search/./sbin/db_mgmt_json.py", line 22, in
from lib.Sources_process import CVEDownloads
File "/opt/emba/external/cve-search/sbin/../lib/Sources_process.py", line 17, in
from lib.DatabaseLayer import (
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseLayer.py", line 35, in
mongo_version = db.command("buildinfo")["versionArray"]
File "/usr/local/lib/python3.9/dist-packages/pymongo/database.py", line 757, in command
with self.__client._socket_for_reads(
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1346, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 244, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 202, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 172.36.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 61af3ed01f494612bdad95e0, topology_type: Single, servers: [<ServerDescription ('172.36.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.36.0.1:27017: timed out')>]>
Traceback (most recent call last):
File "/opt/emba/external/cve-search/./sbin/db_updater.py", line 21, in
from lib.DatabaseSchemaChecker import SchemaChecker
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseSchemaChecker.py", line 7, in
from lib.DatabaseHandler import DatabaseHandler
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseHandler.py", line 4, in
from lib.DatabasePlugins.config import DatabasePluginLoader
File "/opt/emba/external/cve-search/sbin/../lib/DatabasePlugins/config.py", line 1, in
from lib.DatabasePlugins import *
File "/opt/emba/external/cve-search/sbin/../lib/DatabasePlugins/mongodb.py", line 12, in
from lib.DatabaseLayer import sanitize
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseLayer.py", line 35, in
mongo_version = db.command("buildinfo")["versionArray"]
File "/usr/local/lib/python3.9/dist-packages/pymongo/database.py", line 757, in command
with self.__client._socket_for_reads(
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1346, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 244, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 202, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 172.36.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 61af3eef37ab0f6687597923, topology_type: Single, servers: [<ServerDescription ('172.36.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.36.0.1:27017: timed out')>]>
If you are running into cve-search issues please update your EMBA installation:
sudo service mongod stop
sudo /etc/init.d/redis-server stop
sudo /etc/init.d/docker restart
sudo ifconfig emba_runs down
git clone https://github.com/e-m-b-a/emba.git new_emba
cd new_emba
sudo ./installer.sh -d
After testing you should be able to remove your old EMBA installation and the installed mongodb.
With the latest updates (see #183) we have isolated the EMBA container from a networking perspective. This means EMBA is not able to reach the internet anymore. With this also some adjustments in the EMBA docker container and in the local mongodb configuration were needed.
The easiest way to get an updated EMBA installation are the following steps:
cd emba-directory
git pull
sudo docker pull embeddedanalyzer/emba
sudo ./installer.sh -d
If you are running into issues that the EMBA container is not able to access your mongodb installation you can check the following:
ifconfig
should show a new network interface with the following ip address:└─$ ifconfig emba_runs
emba_runs: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.36.0.1 netmask 255.255.0.0 broadcast 172.36.255.255
ether 02:42:cb:c9:37:80 txqueuelen 0 (Ethernet)
RX packets 15643 bytes 964808 (942.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 43695 bytes 1435782652 (1.3 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
If this is not the case you can try to restart docker at all with the following command
└─$ sudo /etc/init.d/docker restart
Another way is to delete the network interface and re-initialize it:
└─$ sudo docker network rm emba_runs
└─$ sudo docker-compose up --no-start
Additionally, you can try the following command
└─$ sudo systemctl restart NetworkManager docker
└─$ grep bindIp /etc/mongod.conf
bindIp: 172.36.0.1
EMBA is complaining that CVE search works not as expected. Checking if mongod is listening, does not return an open port:
└─$ sudo netstat -anpt | grep 27017
Also a restart of mongod does not help!
Please check if your harddisk was running out of space (e.g. with df -h). If it is full, clean it up and retry it.
Check your mongodb log /var/log/mongodb/mongod.log
for errors. One of the seen errors is the following:
{"t":{"$date":"2023-03-24T15:40:07.026+08:00"},"s":"E", "c":"STORAGE", "id":20557, "ctx":"initandlisten","msg":"DBException in initAndListen, terminating","attr":{"error":"IllegalOperation: Attempted to create a lock file on a read-only directory: /var/lib/mongodb"}}
Check this solution and adjust the permissions.
Probably you are running into the issue that your network environment is working as expected and mongod is also listening on port 27017. If you are trying to query your CVE database you get no results:
Sometimes it happens that the CVE database is corrupt or not fully populated. So, please do a full reinitialization and update the following way:
└─$ source ./external/emba_venv/bin/activate
└─$ ./external/cve-search/sbin/db_updater.py -f
As from November 2023 cve-search switched to the new NIST API. From there on you can also use the following command:
└─$ cvexplore database initialize
During a manual CVE database update you are running into the following errors:
It looks like this or this issue.
During a manual CVE database update you are running into the following errors:
CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html', 'connection': 'close'}
Error is gone by adjusting max_workers to 1 in https://github.com/cve-search/cve-search/blob/1f0b50aa46814e2a683a9b7b01da8bcc0403154e/lib/DownloadHandler.py#L121 like this
thread_map(self.download_site, sites, desc="Downloading files", max_workers=1)
Afterwards ./external/cve-search/sbin/db_updater.py -f
should work fine (takes longer though).
Seems like there is a rate limit with https://nvd.nist.gov/
Source: cve-search/cve-search#890 (comment)
thx for documenting this issue to @brainsht
If you get the following warnings you need to update the requests
package.
/usr/local/lib/python3.10/dist-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (5.0.0)/charset_normalizer (2.0.12) doesn't match a supported version!
warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "
Update the requests package the following way should solve this issue:
└─$ source ./external/emba_venv/bin/activate
└─$ sudo pip install -U requests
└─$ sudo docker-compose run emba
WARNING: The FIRMWARE variable is not set. Defaulting to a blank string.
WARNING: The LOG variable is not set. Defaulting to a blank string.
WARNING: The EMBA variable is not set. Defaulting to a blank string.
Creating emba_forked_emba_run ... done
┌──(root@d2a5960b73b2)-[/emba]
└─# source /external/emba_venv/bin/activate
┌──(root@d2a5960b73b2)-[/emba]
└─# /external/cve-search/bin/search.py -p busybox
This should show some CVE details regarding busybox.
Is your feature request related to a problem? Please describe.
"Modern" android OTA updates contain a file called payload.bin that start with a magic "CrAU"
While emba does find some part of the file it would be nice to have full support for this (common) file format
Describe the solution you'd like
Starting from a zip obtained from
https://developers.google.com/android/ota
oriole-ota-sd1a.210817.015.a4-19a77b62.zip
https://dl.google.com/dl/android/aosp/oriole-ota-sd1a.210817.015.a4-19a77b62.zip
git clone https://github.com/vm03/payload_dumper.git
cd payload_dumper
pip install -r requirements.txt
unzip ../oriole-ota-sd1a.210817.015.a4-19a77b62.zip
python payload_dumper.py payload.bin
Processing system partition..........................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing system_ext partition..........................................................................................................................Done
Processing product partition.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing vbmeta_system partition.Done
Processing boot partition................................Done
Processing vendor_boot partition................................Done
Processing dtbo partition........Done
Processing vbmeta partition.Done
Processing vbmeta_vendor partition.Done
Processing vendor partition......................................................................................................................................................................................................................................Done
Processing vendor_dlkm partition...........................Done
Processing bl1 partition.Done
Processing pbl partition.Done
Processing bl2 partition.Done
Processing abl partition.Done
Processing bl31 partition.Done
Processing tzsw partition...Done
Processing gsa partition.Done
Processing ldfw partition..Done
Processing modem partition...................................Done
The result is found in output
file output/*.img
output/bl1.img: data
output/bl2.img: data
output/bl31.img: data
output/boot.img: Android bootimg, kernel (0x150d94), ramdisk (0x630)
output/dtbo.img: data
output/gsa.img: data
output/ldfw.img: data
output/modem.img: POSIX tar archive (GNU)
output/pbl.img: Dyalog APL version 165.7
output/product.img: Linux rev 1.0 ext2 filesystem data, UUID=8e66e769-b7a9-574e-b7d2-513c40dbb996, volume name "product" (extents) (large files) (huge files)
output/system_ext.img: Linux rev 1.0 ext2 filesystem data, UUID=fc31cba1-4585-5da0-9700-ecedd28b80ec, volume name "system_ext" (extents) (large files) (huge files)
output/system.img: Linux rev 1.0 ext2 filesystem data, UUID=02e80408-f118-591d-90f7-5b2411e5859b (extents) (large files) (huge files)
output/tzsw.img: data
output/vbmeta.img: data
output/vbmeta_system.img: data
output/vbmeta_vendor.img: data
output/vendor_boot.img: data
output/vendor_dlkm.img: Linux rev 1.0 ext2 filesystem data, UUID=d550f889-ddd7-5920-bf31-ffd8c5dee97e, volume name "vendor_dlkm" (extents) (large files) (huge files)
output/vendor.img: Linux rev 1.0 ext2 filesystem data, UUID=d8891240-d867-5fac-80a5-6e9859e0263d, volume name "vendor" (extents) (large files) (huge files)
It would be nice to integrate this .
Describe alternatives you've considered
Manual work
Additional context
https://www.thecustomdroid.com/how-to-extract-android-payload-bin-file/
Describe the bug
Installation failed
To Reproduce
Steps to reproduce the behavior:
clone repo
run: sudo ./installer.sh -d
Expected behavior
Expected no error
Error detected - status code 1
Command: docker-compose up --no-start
Location: ./installer/I05_emba_docker_image_dl.sh, line 49
Stack Trace:
[1] I05_emba_docker_image_dl(): ./installer/I05_emba_docker_image_dl.sh, line 49 -> I05_emba_docker_image_dl
[2] main(): ./installer.sh, line 152 -> main -d
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
Add a -F or -Y switch to installer.sh to force an install of everything, even if its optional.
Currently the installer only shows the dpkg deps and some others. It is not consistent and does not show our pip deps, our deps from cve-search and binwalk. We should show them all in the future
What are you missing in emba? Let us know how you are using emba, what is working and what doesn't work
From our point of view we have the following areas to improve:
What else would be great to bring to emba?
our cwe-checker integration has some troubles running from dockerized emba
Multiple issues found by reddit user geirha https://www.reddit.com/r/bash/comments/o2ib89/emba_an_open_source_firmware_analyzer_has/
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.