Typical cve-search issues about emba

[aserper]

In my experience, trying to install EMBA on a server with a lot of cores (I have it running on a 36 core aws machine right now) causes the same issues with updating and populating the CVE database. A workaround that I found to that issue is to run the emba installer solely by utilizing two cores. While I didn't really look into why it helps, I assume that the large number of cores opens more threads which causes some DDOS protection mechanism on the remote server to kick in (again, just a guess). This can be done by running the installer program like so:
sudo taskset -c 0,1 ./installer.sh -d which will run the installation script using only two cores. I know that you can go into the files and change the code but I'm lazy ;)

[brainsht]

Found another source of cve-serach issues. Kept getting this error message while running ./external/cve-search/sbin/db_updater.py -f

CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html', 'connection': 'close'}

Error is gone by adjusting max_workers to 1 in https://github.com/cve-search/cve-search/blob/1f0b50aa46814e2a683a9b7b01da8bcc0403154e/lib/DownloadHandler.py#L121 like this

thread_map(self.download_site, sites, desc="Downloading files", max_workers=1)

Afterwards ./external/cve-search/sbin/db_updater.py -f works fine (takes longer though) and no cve-serach issue anymore for me.

Really seems like there is a rate limit with https://nvd.nist.gov/
Source: cve-search/cve-search#890 (comment)

[m-1-k-3]

@brainsht thank you for documenting this issue. I will include it into the main post.

[brainstorm]

thread_map(self.download_site, sites, desc="Downloading files", max_workers=1)

Really seems like there is a rate limit with https://nvd.nist.gov/ Source: cve-search/cve-search#890 (comment)

That max_workers argument is now exposed as $HOME/.cvexplore/.env as MAX_DOWNLOAD_WORKERS and you can define it along with the NVD_NIST_API_KEY variable.

sudo taskset -c 0,1 ./installer.sh -d is not yet documented in the wiki and I suspect that there are still threading issues within EMBA, probably (see: #908 (comment)).

[m-1-k-3]

If you are running into cve-search issues please test #913

[github-actions]

This issue is stale because it has been open for 14 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale.

[Hellworold]

Hi,
When iam trying to run the utility it gives an error CVE- search not working , check issue number 187 thus iam here .
Now i have checked all the steps above.
for some reason this is the output
i have checked with some older versions

[m-1-k-3]

Please check the following:

• is your network ok? -> ifconfig should show the emba_runs network device with ip address 172.36.0.1
• is mongod running? netstat -anpt | grep 27017
• is redis running? netstat -anpt | grep 6379

[m-1-k-3]

Then check mongod, restart it, check logs and as it is running you can do an update of the database.

[Darren-zl]

I follow the above operations to do, are normal. But I still show cve-search -no ok. I don't know why.

[m-1-k-3]

@aserper thank you for documenting this issue and the workaround. I will add a link to the wiki

[m-1-k-3]

With version 1.3.0 we switched to python environment. This results in the need of activating the python environment before doing cve-search tasks.

On the host system it is possible to activate it the following way:

└─$ source ./external/emba_venv/bin/activate

In the docker container it is possible to activate it the following way:

└─$ source /external/emba_venv/bin/activate

[m-1-k-3]

It is now recommend to setup a NIST API key
See also https://github.com/e-m-b-a/emba/wiki/Installation#prerequisites