GithubHelp home page GithubHelp logo

csirtgadgets / cif-v5 Goto Github PK

View Code? Open in Web Editor NEW
62.0 7.0 11.0 135 KB

The FASTEST way to consume threat intel.

Home Page: https://csirtgadgets.com

License: Mozilla Public License 2.0

Makefile 0.31% Python 94.53% Shell 3.31% Dockerfile 1.85%
cif threatintel threathunting csirtg

cif-v5's People

Contributors

drawks avatar jtkristoff avatar therealpancakes avatar wesyoung avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar

Forkers

sfinlon newbdoc cloudware-llc tonyphipps drawks belutak syllogy therealpancakes sh9369 darksidesfear

cif-v5's Issues

CIF python client, module not found

Ubuntu 16 LTS

After installing the Python client: pip install 'cifsdk>=5.0b1,<6.0' (also tried 5.0b4), runnig the cif command results in the following error message.
Capture
It states that it can't find the 'geoip2' module, although all requirements are satisfied.

Figured i'd at least post this in case this wasn't a known issue. It also may be nothing and completely on my end so feel free to dismiss.

CIF client parameter '--limit' doesn't limit the number of results returned

Hello,

The '--limit' parameter doesn't limit the numer of results returned when using cif client. I'm using CIFv5 with a ElasticSearch 7.

/usr/local/bin/cif --itype fqdn --confidence 2 --tags botnet --limit 5 -d -v

2020-07-07 19:34:09,295 - DEBUG - urllib3.connectionpool[230][MainThread] - Starting new HTTP connection (1): localhost:5000
2020-07-07 19:34:09,986 - DEBUG - urllib3.connectionpool[442][MainThread] - http://localhost:5000 "GET /indicators?itype=fqdn&confidence=2&limit=5&tags=botnet HTTP/1.1" 200 115264
+-------+----------+----------------------------+----------------------------------+-----+----+----------------------------+----------------------------+-------+--------------+-------------------+------------+-----------------------------+------------------------------+-------------+----------------------------------+
| tlp | group | reported_at | indicator | asn | cc | first_at | last_at | count | tags | description | confidence | rdata | provider | probability | reference |
+-------+----------+----------------------------+----------------------------------+-----+----+----------------------------+----------------------------+-------+--------------+-------------------+------------+-----------------------------+------------------------------+-------------+----------------------------------+
| amber | everyone | 2020-07-07T02:24:00.00000Z | restaaojrplinlm.de | | | | | 1 | botnet | 2020-07-07 01:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-07T01:24:00.00000Z | xnebnulmkngu.de | | | | | 1 | botnet | 2020-07-07 00:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-07T01:24:00.00000Z | ifxgfhxqlnkhsrdbmlxdy.de | | | | | 1 | botnet | 2020-07-07 00:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-07T01:24:00.00000Z | evphygiwubge.de | | | | | 1 | botnet | 2020-07-07 00:14 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-06T03:44:40.00000Z | girhrbfbggtsvcl2h.com | | | | | 1 | botnet | 2020-07-06 03:05 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-05T05:44:40.00000Z | sejehjtqvihbm.com | | | | | 1 | botnet | 2020-07-05 04:20 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-05T05:44:40.00000Z | qbvxkwoxtdhnaxk.com | | | | | 1 | botnet | 2020-07-05 04:20 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
| amber | everyone | 2020-07-05T05:44:40.00000Z | navvjqbilvdmipwdm.com | | | | | 1 | botnet | 2020-07-05 04:20 | 3 | | osint.bambenekconsulting.com | | http://osint.bambenekconsultin.. |
......

(more results are displayed)

The same occurs if I set --limit to more than 500, only 500 results are returned.

Using CURL instead of CIF client shows the same error:

curl -XGET http://localhost:5000/indicators?itype=fqdn&confidence=2&limit=5&tags=botnet

Thank you,
Jose

Expiration of Indicators of Compromise (IoCs)

Do you have a support subscription?

No

Did you check the FAQ ?

Yes

Expected behavior and actual behavior.

N/A

Question:

Does CIFv5 API or CIFv3 API (which is now archived) have support for indicator expiration? If so, any insight as to how you handle IoC expiration and if there's a field that exposes whether an indicator is expired or if there's a TTL on each IoC etc. will be appreciated.

CIF REST API 500 errors

After working through some of the updates you provided, I've got the CIF python cli working awesome on the CIFV5 server, and everything is humming along great. Except for the fact that the REST api continues to respond with 500 server errors when sending SWAGGER or local curl based requests. An example is:

curl -X GET "http://localhost:5000/indicators?indicator=example.com" -H "accept: application/json"

responds with:

{
  "message": "Internal Server Error"
}

While the cli running:

cif -nq example.com

responds with two entries.

Now, after looking through the cli python code, I attempted to recreate the cli api call and came up with something:

curl -X GET "http://localhost:5000/indicators?indicator=example.com" -H "accept: application/json" -H "User-Agent: cifsdk-py/5"

which does actually return a single entry. Sadly, it's only a tags: search entry while the CIF CLI is returning a freemal,spam entry from github----wesbos. So not quite returning everything but hey, it's not returning an error.

i had many difficulties with the installation of the cif-v5 , can you give me guidance to install it locally step by step ?

Do you have a support subscription?

https://csirtg.io/support

Did you check the FAQ https://github.com/csirtgadgets/cif-v5/wiki/FAQ ?

Expected behavior and actual behavior.

Steps to reproduce the problem

Relevant logs as a result of the actual behavior

https://github.com/csirtgadgets/cif-v5/wiki/FAQ#searching-logs

Did you attempt to fix the problem and submit a pull request?

Specifications like the version of the project, operating system, or hardware.

Does adding additional memory to the box resolve the problem?

How large is your /var/lib/cif.db database?

cif-router resource temporarily unavailable

It would appear there may be an issue with the cif-router docker image and ZMQ. On clean Debian 10 and Ubuntu 20.04 systems (just the base install and Docker) the cifv5.db database never populates with the downloaded data (left overnight and still 152KB) . I added abuse_ch.yml and emerging_threats.yml to the data/rules folder and it appears the files get downloaded, but never get stored in the database. The standard log output as well as with the trace environment variables are below.

Standard log out:
csirtg-fm | 2021-05-18 03:13:37,850 - INFO - csirtg_fm[125] - sending: 500
cif-router | 2021-05-18 03:13:42,950 - ERROR - cif.router.message[34][MainThread] - Resource temporarily unavailable
cif-router | Traceback (most recent call last):
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0-py3.7.egg/cif/router/message.py", line 31, in handle
cif-router | handler(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0-py3.7.egg/cif/router/message.py", line 61, in handle_indicators_create
cif-router | self.enrichment.socket.send_msg(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/cifsdk/zmq/socket.py", line 21, in send_msg
cif-router | return self.send_multipart(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 445, in send_multipart
cif-router | self.send(msg, SNDMORE|flags, copy=copy, track=track)
cif-router | File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 400, in send
cif-router | return super(Socket, self).send(data, flags=flags, copy=copy, track=track)
cif-router | File "zmq/backend/cython/socket.pyx", line 728, in zmq.backend.cython.socket.Socket.send
cif-router | File "zmq/backend/cython/socket.pyx", line 775, in zmq.backend.cython.socket.Socket.send
cif-router | File "zmq/backend/cython/socket.pyx", line 247, in zmq.backend.cython.socket._send_copy
cif-router | File "zmq/backend/cython/socket.pyx", line 242, in zmq.backend.cython.socket._send_copy
cif-router | File "zmq/backend/cython/checkrc.pxd", line 20, in zmq.backend.cython.checkrc._check_rc
cif-router | zmq.error.Again: Resource temporarily unavailable

Debug log out:

cif-router | 2021-05-18 11:12:48,182 - INFO - cif.router[185][MainThread] - loglevel: 10
cif-router | 2021-05-18 11:12:48,186 - DEBUG - cif.router[198][MainThread] - pid: 10
cif-router | 2021-05-18 11:12:48,525 - INFO - cif.router[204][MainThread] - starting router..
cif-router | 2021-05-18 11:12:48,526 - INFO - cif.router[112][MainThread] - launching backend..
cif-router | 2021-05-18 11:12:48,526 - INFO - cif.router[68][MainThread] - launching store...
cif-router | 2021-05-18 11:12:49,435 - INFO - cif.router[75][MainThread] - Waiting for Store to initialize...
cif-router | 2021-05-18 11:12:51,438 - INFO - cif.router[77][MainThread] - Store Ready....
cif-router | 2021-05-18 11:12:51,439 - INFO - cif.router[115][MainThread] - launching frontend...
cif-router | 2021-05-18 11:12:51,439 - INFO - cif.router[118][MainThread] - listening on: ipc:///var/lib/cif/router.ipc
cif-enrichers exited with code 1
csirtg-fm | 2021-05-18 11:13:49,636 - INFO - csirtg_fm.cli[85] - starting run...
csirtg-fm | 2021-05-18 11:13:49,686 - INFO - csirtg_fm.cli[157] - processing: abuse_ch.yml - urlhaus
csirtg-fm | 2021-05-18 11:14:18,478 - INFO - csirtg_fm[125] - sending: 500
cif-router | 2021-05-18 11:14:18,529 - DEBUG - cif.router.message[28][MainThread] - handling message: indicators_create
cif-router | 2021-05-18 11:14:18,530 - DEBUG - cif.router.message[57][MainThread] - messages: 500
cif-router | 2021-05-18 11:14:23,536 - ERROR - cif.router.message[34][MainThread] - Resource temporarily unavailable
cif-router | Traceback (most recent call last):
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0-py3.7.egg/cif/router/message.py", line 31, in handle
cif-router | handler(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0-py3.7.egg/cif/router/message.py", line 61, in handle_indicators_create
cif-router | self.enrichment.socket.send_msg(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/cifsdk/zmq/socket.py", line 21, in send_msg
cif-router | return self.send_multipart(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 445, in send_multipart
cif-router | self.send(msg, SNDMORE|flags, copy=copy, track=track)
cif-router | File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 400, in send
cif-router | return super(Socket, self).send(data, flags=flags, copy=copy, track=track)
cif-router | File "zmq/backend/cython/socket.pyx", line 728, in zmq.backend.cython.socket.Socket.send
cif-router | File "zmq/backend/cython/socket.pyx", line 775, in zmq.backend.cython.socket.Socket.send
cif-router | File "zmq/backend/cython/socket.pyx", line 247, in zmq.backend.cython.socket._send_copy
cif-router | File "zmq/backend/cython/socket.pyx", line 242, in zmq.backend.cython.socket._send_copy
cif-router | File "zmq/backend/cython/checkrc.pxd", line 20, in zmq.backend.cython.checkrc._check_rc
cif-router | zmq.error.Again: Resource temporarily unavailable
cif-router | Traceback (most recent call last):
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0-py3.7.egg/cif/router/message.py", line 31, in handle
cif-router | handler(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/cif-5.0-py3.7.egg/cif/router/message.py", line 61, in handle_indicators_create
cif-router | self.enrichment.socket.send_msg(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/cifsdk/zmq/socket.py", line 21, in send_msg
cif-router | return self.send_multipart(m)
cif-router | File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 445, in send_multipart
cif-router | self.send(msg, SNDMORE|flags, copy=copy, track=track)
cif-router | File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 400, in send
cif-router | return super(Socket, self).send(data, flags=flags, copy=copy, track=track)
cif-router | File "zmq/backend/cython/socket.pyx", line 728, in zmq.backend.cython.socket.Socket.send
cif-router | File "zmq/backend/cython/socket.pyx", line 775, in zmq.backend.cython.socket.Socket.send
cif-router | File "zmq/backend/cython/socket.pyx", line 247, in zmq.backend.cython.socket._send_copy
cif-router | File "zmq/backend/cython/socket.pyx", line 242, in zmq.backend.cython.socket._send_copy
cif-router | File "zmq/backend/cython/checkrc.pxd", line 20, in zmq.backend.cython.checkrc._check_rc
cif-router | zmq.error.Again: Resource temporarily unavailable
cif-router | 2021-05-18 11:14:23,539 - DEBUG - cif.router.message[35][MainThread] - NoneType

is the problem of elastic search still persist ?

Do you have a support subscription?

https://csirtg.io/support

Did you check the FAQ https://github.com/csirtgadgets/cif-v5/wiki/FAQ ?

Expected behavior and actual behavior.

Steps to reproduce the problem

Relevant logs as a result of the actual behavior

https://github.com/csirtgadgets/cif-v5/wiki/FAQ#searching-logs

Did you attempt to fix the problem and submit a pull request?

Specifications like the version of the project, operating system, or hardware.

Does adding additional memory to the box resolve the problem?

How large is your /var/lib/cif.db database?

Large time-based feed calls crash

Do you have a support subscription?

yes, fortifydata

Did you check the FAQ https://github.com/csirtgadgets/cif-v5/wiki/FAQ ?

I made sure that my memory and storage were at acceptable levels for the docker stats

Expected behavior and actual behavior.

curl -X GET "http://localhost:5000/indicators?limit=10000&itype=ipv4&hours=1&nolog=1&confidence=3" -H "accept: application/json"

When running the previous command, either through curl/swagger/ or the Python CLI (with same args), the logs show that the CIF server is crashing while trying to process the returning data that is requested. By pulling the database, I can see that requesting the past hour should give me ~2k-3k records. The database is sitting at 218M. It would be expected that those records are returned.

As a workaround, I have found that using the /Indicators POST endpoint actually works (Although you skip the benefits of the whitelist I think? Right?) . The downside of using the post endpoint is that you dont get the automatic "reported_at" processing on hours/days/etc. So if you duplicate the "calc_reported_at_window" function and generate the "reported_at" on the ingesting side, it will actually spit out those expcted ~2k-3k results that can be verified in the database.

curl -X POST "http://3.21.225.128:5000/indicators" -H "accept: application/json" -H "Content-Type: application/json" -d "[ { \"itype\": \"ipv4\", \"confidence\": 3, \"limit\": 50000, \"nolog\": 1, \"reported_at\": \"2020-03-24T11:11:20,2020-03-24T12:41:20\" }]"

Steps to reproduce the problem

  1. Spin up your CIF server and let it run for a few days in order to build a large database.
  2. Using the CIF CLI or CURL or SWAGGER, request a feed of the past hour with ipv4, confidence of 3, limit of 10000 and nolog = 1.
  3. Check the logs

Relevant logs as a result of the actual behavior

cif-httpd       | [2020-03-24 12:49:01,870] ERROR in app: Exception on /indicators [GET]
cif-httpd       | Traceback (most recent call last):
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 22, in _get
cif-httpd       |     r = client.indicators_search(filters)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/client/zeromq/__init__.py", line 22, in indicators_search
cif-httpd       |     decode=decode)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/client/zeromq/base.py", line 198, in _send
cif-httpd       |     return self._recv(decode=decode, close=self.autoclose)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/client/zeromq/base.py", line 161, in _recv
cif-httpd       |     m = self.socket.recv_msg()
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/zmq/socket.py", line 24, in recv_msg
cif-httpd       |     m = self.recv_multipart()
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 475, in recv_multipart
cif-httpd       |     parts = [self.recv(flags, copy=copy, track=track)]
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 791, in zmq.backend.cython.socket.Socket.recv
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 827, in zmq.backend.cython.socket.Socket.recv
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 191, in zmq.backend.cython.socket._recv_copy
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 186, in zmq.backend.cython.socket._recv_copy
cif-httpd       |   File "zmq/backend/cython/checkrc.pxd", line 20, in zmq.backend.cython.checkrc._check_rc
cif-httpd       | zmq.error.Again: Resource temporarily unavailable
cif-httpd       |
cif-httpd       | During handling of the above exception, another exception occurred:
cif-httpd       |
cif-httpd       | Traceback (most recent call last):
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/indicators.py", line 130, in get
cif-httpd       |     rv = get_feed(f)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 60, in get_feed
cif-httpd       |     _get_whitelist(filters)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 80, in _get_whitelist
cif-httpd       |     return aggregate(_get(wl_filters))
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 25, in _get
cif-httpd       |     raise ConnectionError
cif-httpd       | ConnectionError
cif-httpd       |
cif-httpd       | During handling of the above exception, another exception occurred:
cif-httpd       |
cif-httpd       | Traceback (most recent call last):
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
cif-httpd       |     rv = self.dispatch_request()
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
cif-httpd       |     return self.view_functions[rule.endpoint](**req.view_args)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/api.py", line 325, in wrapper
cif-httpd       |     resp = resource(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask/views.py", line 89, in view
cif-httpd       |     return self.dispatch_request(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/resource.py", line 44, in dispatch_request
cif-httpd       |     resp = meth(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/indicators.py", line 143, in get
cif-httpd       |     api.abort(503)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/namespace.py", line 141, in abort
cif-httpd       |     abort(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/errors.py", line 31, in abort
cif-httpd       |     flask.abort(code)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/werkzeug/exceptions.py", line 772, in abort
cif-httpd       |     return _aborter(status, *args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/werkzeug/exceptions.py", line 753, in __call__
cif-httpd       |     raise self.mapping[code](*args, **kwargs)
cif-httpd       | werkzeug.exceptions.ServiceUnavailable: 503 Service Unavailable: The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
cif-httpd       | [2020-03-24 12:49:01,870] ERROR in app: Exception on /indicators [GET]
cif-httpd       | Traceback (most recent call last):
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 22, in _get
cif-httpd       |     r = client.indicators_search(filters)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/client/zeromq/__init__.py", line 22, in indicators_search
cif-httpd       |     decode=decode)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/client/zeromq/base.py", line 198, in _send
cif-httpd       |     return self._recv(decode=decode, close=self.autoclose)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/client/zeromq/base.py", line 161, in _recv
cif-httpd       |     m = self.socket.recv_msg()
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cifsdk/zmq/socket.py", line 24, in recv_msg
cif-httpd       |     m = self.recv_multipart()
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/zmq/sugar/socket.py", line 475, in recv_multipart
cif-httpd       |     parts = [self.recv(flags, copy=copy, track=track)]
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 791, in zmq.backend.cython.socket.Socket.recv
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 827, in zmq.backend.cython.socket.Socket.recv
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 191, in zmq.backend.cython.socket._recv_copy
cif-httpd       |   File "zmq/backend/cython/socket.pyx", line 186, in zmq.backend.cython.socket._recv_copy
cif-httpd       |   File "zmq/backend/cython/checkrc.pxd", line 20, in zmq.backend.cython.checkrc._check_rc
cif-httpd       | zmq.error.Again: Resource temporarily unavailable
cif-httpd       |
cif-httpd       | During handling of the above exception, another exception occurred:
cif-httpd       |
cif-httpd       | Traceback (most recent call last):
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/indicators.py", line 130, in get
cif-httpd       |     rv = get_feed(f)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 60, in get_feed
cif-httpd       |     _get_whitelist(filters)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 80, in _get_whitelist
cif-httpd       |     return aggregate(_get(wl_filters))
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/feeds/__init__.py", line 25, in _get
cif-httpd       |     raise ConnectionError
cif-httpd       | ConnectionError
cif-httpd       |
cif-httpd       | During handling of the above exception, another exception occurred:
cif-httpd       |
cif-httpd       | Traceback (most recent call last):
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
cif-httpd       |     rv = self.dispatch_request()
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
cif-httpd       |     return self.view_functions[rule.endpoint](**req.view_args)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/api.py", line 325, in wrapper
cif-httpd       |     resp = resource(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask/views.py", line 89, in view
cif-httpd       |     return self.dispatch_request(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/resource.py", line 44, in dispatch_request
cif-httpd       |     resp = meth(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/http/indicators.py", line 143, in get
cif-httpd       |     api.abort(503)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/namespace.py", line 141, in abort
cif-httpd       |     abort(*args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/flask_restplus/errors.py", line 31, in abort
cif-httpd       |     flask.abort(code)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/werkzeug/exceptions.py", line 772, in abort
cif-httpd       |     return _aborter(status, *args, **kwargs)
cif-httpd       |   File "/usr/local/lib/python3.7/site-packages/werkzeug/exceptions.py", line 753, in __call__
cif-httpd       |     raise self.mapping[code](*args, **kwargs)
cif-httpd       | werkzeug.exceptions.ServiceUnavailable: 503 Service Unavailable: The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
```

# Did you attempt to fix the problem and submit a pull request?

I traced the get call all through the code and found that it seems to be a ZMQ error. When the:
get_feed -> _get_whitelist -> _get call is made, the error is being generated in the http.feeds._get method as the ZMQ seems to be having a ConnectionError. I'm not super familiar with Python messaging queues, but maybe a bandwidth issue? I dunno.

# Specifications like the version of the project, operating system, or hardware.
Ubuntu 18LTS, AWS EC2 t2.large, 2 cpu, 8gb rm, 25gb storage.

# How large is your /var/lib/cif.db database?
~230M

setup.py's custom python version check doesn't allow cutting dist tarballs using arbitrary python version

I'm just doing some work around packaging this software for use in some internal infrastructure and ran across the check in setup.py which causes it to bail unless being run on python 3.7. All modern versions of setuptools allow you to specify the python requirements of the software in the metadata, which allows for you to do things like cut release tarballs without satisfying the requirement, since naturally you don't actually need to run the software to cut a release.

Really this is just a minor annoyance, but I'd prefer not to maintain a separate fork just to have this one check disabled.

Look out for incoming MR

Search indicators in Bulk fails

Do you have a support subscription?

yes: fortifydata

Expected behavior and actual behavior.

When using the rest api, the /indicators POST endpoint should be able to run bulk search queries for multiple indicator searches. However, the query fails when you add more than one indicator to search for.

From what I tracked through the code, it seems to be the code is looking for a "limit" attribute on the list object passed in? So maybe the search aspect needs to check whether the search data is a list or a single object?

Steps to reproduce the problem

Spin up cif and either using CURL or the swagger endpoint, attempt to make a POST bulk search for more than one indicator.

An example curl is below.

curl -X POST "http://3.21.225.128:5000/indicators" -H "accept: application/json" -H "Content-Type: application/json" -d "[ { \"indicator\": \"88.15.65.0\", \"itype\": \"ipv4\", \"limit\": 500, \"nolog\": \"1\" }, { \"indicator\": \"124.118.197.129\", \"itype\": \"ipv4\", \"limit\": 500, \"nolog\": \"1\" }]"

As can be seen, I experimented with adding the "limit" parameter in there, as that seems to be causing the issues?

Relevant logs as a result of the actual behavior

cif-router      | 2020-03-23 17:14:51,656 - ERROR - cif.store.handlers.indicator[93][MainThread] - 'list' object has no attribute 'limit'
cif-router      | 2020-03-23 17:14:51,656 - ERROR - cif.store[109][MainThread] - invalid search
cif-router      | Traceback (most recent call last):
cif-router      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/handlers/indicator.py", line 87, in indicators_search
cif-router      |     yield from self.store.indicators.search(m.data)
cif-router      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/sqlite/indicator.py", line 83, in search
cif-router      |     return self._search_bulk(filters).limit(500)
cif-router      | AttributeError: 'list' object has no attribute 'limit'
cif-router      |
cif-router      | During handling of the above exception, another exception occurred:
cif-router      |
cif-router      | Traceback (most recent call last):
cif-router      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/__init__.py", line 103, in _trigger_handler
cif-router      |     m.data = json.dumps(rv)
cif-router      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/store/handlers/indicator.py", line 99, in indicators_search
cif-router      |     raise TypeError('invalid search')
cif-router      | TypeError: invalid search

Specifications like the version of the project, operating system, or hardware.

Latest version, Ubuntu 18LTS, AWS EC2 t2.large, 2 cpus, 8gb ram, 25gb storage

How large is your /var/lib/cif.db database?

302M?

It would be nice if the default rules were baked into the cif package

I'll preface this by saying I understand that your repository is setup to support installation as an orchestrated set of docker containers. However, this note in your README The default, CIF/Docker configuration is NOT meant to be deployed in large scale operations. That's your job. leads me to believe that it is desirable for all the components to be easily composable. So, I'm left wondering why not make the rules subdirectory part of the cif python package by way of setuptools package data support? Unless I'm misunderstanding something, this default rule set is mean to be expanded upon/overridden by pointing csirtg-fm at additional directories of "local_rules". It sure would be nice if these default upstream rules were included when cutting sdist tarball. I see that they are specifically excluded in MANIFEST.in which leaves me thinking perhaps I'm misunderstanding the intent here.

Any information would be greatly appreciated, I'd be more than happen to offer up an MR that bundles the default rules into the package data and also updates the code to use the platform independent lookups for these bundled resources. I think ultimately it would remove a few steps from your dockercompose and it would make the default rules a bit more baked in for those of us who are trying to use these python modules independent of your reference deployment.

flask-restplus should be dropped in favor of flask-restx

While doing some testing of the CIFv5 stack I came across an exception in cif-httpd

Traceback (most recent call last):
  File "/usr/local/bin/cif-httpd", line 10, in <module>
    from cif.http.app import main
  File "/venv/cifv5/lib/python3.6/site-packages/cif/http/app.py", line 13, in <module>
    from flask_restplus import Api
  File "/venv/lib/python3.6/site-packages/flask_restplus/__init__.py", line 5, in <module>
    from .api import Api  # noqa
  File "/venv/cifv5/lib/python3.6/site-packages/flask_restplus/api.py", line 24, in <module>
    from werkzeug import cached_property
ImportError: cannot import name 'cached_property'

It looks like the flask-restplus module depends on an older version of werkzeug, not entirely unexpected as the version is pinned downrev to 0.16 in the requirements.txt. However when I looked a little bit further it appears that flask-restplus has been fully abandoned and the maintainers of the project have started releasing under a new forked project

It would be a good idea to update CIFv5 to use this newer module which would bring compatibility with the current release of werkzeug.

API based search doesn't return known entries

Do you have a support subscription?

Yes, fortifydata

Did you check the FAQ https://github.com/csirtgadgets/cif-v5/wiki/FAQ ?

Yes.

Expected behavior and actual behavior.

It would be expected that searching for an indicator through the REST api (indicator=? or q=?) would return a non-search entry when the provided indicator has been confirmed to be in the system and in the database. Instead, grabbing an ipv4/url indicator from a returned feed or from the sqlite database and searching for it with the REST api only returns a tag:search entry. This behavior is present in the GET (single) indicators as well as the POST (bulk) indicators call.

I have confirmed that running the CIF python client and using -nq, results are as expected

cif -nq 111.42.102.68 (or any known indicator)

Running the standard -q results in only the search tag entry

cif -q 111.42.102.68 (or any known indicator)

Steps to reproduce the problem

After CIFv5 has been running for a few hours do one of the following:

  1. use the rest api to get a list of indicators filtered by tags and itype, and copy a known ipv4/url indicator
  2. pull down a copy of the sqlite database, find a random row, and copy the ipv4/url indicator

Then, using the REST API, search for that ipv4/url indicator using q=? or indicator=? (I tried both).
Whether you use the GET or POST /indicators, the results are the same

One entry is returned, with a tag of search

Relevant logs as a result of the actual behavior

I'm still tweaking the hunters, and am only using 1 thread which may be the timeouts.

ence": "https://urlhaus.abuse.ch/url/324553/","rdata": "http://125.40.33.61:34227/mozi.m","last_at": "2020-03-20T19:02:31.959804Z","reported_at": "2020-03-20T19:02:31.959814Z"}
cif-hunter      | 2020-03-20 19:43:12,312 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:43:12,313 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "182.116.78.197","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "2c50fab3-c466-4a4d-a63d-e1862c50ab50","iid": "e1b31d95-5f7a-4c85-bfe6-46b4276af08b","reference": "https://urlhaus.abuse.ch/url/324551/","rdata": "http://182.116.78.197:41785/mozi.m","last_at": "2020-03-20T19:02:31.961695Z","reported_at": "2020-03-20T19:02:31.961706Z"}
cif-router      | 2020-03-20 19:43:17,132 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:22,138 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:27,143 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:32,144 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:37,150 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:42,153 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:43:42,370 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:43:42,409 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:43:42,410 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "124.229.173.159","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "7d079dc5-e13b-44e8-bb50-a6949d96e16c","iid": "b465c844-4404-4b64-83e4-a3efbeab4543","reference": "https://urlhaus.abuse.ch/url/324549/","rdata": "http://124.229.173.159:55650/mozi.m","last_at": "2020-03-20T19:02:31.963576Z","reported_at": "2020-03-20T19:02:31.963586Z"}
cif-hunter      | 2020-03-20 19:43:42,479 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:43:42,479 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "219.155.173.255","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "4e9a2278-be99-4866-9682-1b1d9454dbea","iid": "aaacf0df-e113-4ffe-86fc-d210aa4d2430","reference": "https://urlhaus.abuse.ch/url/324547/","rdata": "http://219.155.173.255:57184/mozi.m","last_at": "2020-03-20T19:02:31.965500Z","reported_at": "2020-03-20T19:02:31.965510Z"}
cif-router      | 2020-03-20 19:43:47,154 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:52,160 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:43:57,166 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:02,167 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:07,173 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:12,174 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:44:12,509 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:44:12,576 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:44:12,577 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "1.246.222.80","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "c4b4ec9b-5440-4113-9eb3-4b6b7e6f3ac2","iid": "96378bc5-8200-4337-af31-320a4d352bc5","reference": "https://urlhaus.abuse.ch/url/324545/","rdata": "http://1.246.222.80:4160/mozi.m","last_at": "2020-03-20T19:02:31.967375Z","reported_at": "2020-03-20T19:02:31.967386Z"}
cif-router      | 2020-03-20 19:44:17,180 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:22,185 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:27,187 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:32,190 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:37,195 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:42,201 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:44:42,647 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:44:47,202 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:52,203 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:44:57,209 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:02,214 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:07,220 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:12,222 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:45:12,693 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:45:17,225 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:22,227 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:27,232 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:32,236 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:37,241 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:42,244 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:45:42,745 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:45:42,776 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:45:42,777 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "111.42.66.93","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "f9315252-a4d6-49e4-afac-a55b227109ba","iid": "2f0677e4-2b6d-4469-8912-5a5f2347cf4d","reference": "https://urlhaus.abuse.ch/url/324541/","rdata": "http://111.42.66.93:35722/mozi.m","last_at": "2020-03-20T19:02:31.971343Z","reported_at": "2020-03-20T19:02:31.971354Z"}
cif-router      | 2020-03-20 19:45:47,247 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:52,249 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:45:57,254 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:02,255 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:07,261 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:12,266 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:46:12,820 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:46:17,272 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:22,277 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:27,283 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:32,289 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:37,293 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:42,296 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:46:42,866 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:46:42,895 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:46:42,895 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "111.42.66.41","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "66b1bfad-257f-4379-bccc-19e0cb3e1db1","iid": "ee746712-4e6f-447f-979b-eb511977596c","reference": "https://urlhaus.abuse.ch/url/324538/","rdata": "http://111.42.66.41:52419/mozi.m","last_at": "2020-03-20T19:02:31.974177Z","reported_at": "2020-03-20T19:02:31.974187Z"}
cif-router      | 2020-03-20 19:46:47,302 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:52,305 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:46:57,307 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:02,311 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:07,317 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:12,321 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:47:12,958 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:47:12,992 - ERROR - csirtg_indicator.wrappers.geo[17][MainThread] - maxmind data/libraries not installed
cif-router      | 2020-03-20 19:47:17,326 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:22,331 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:27,336 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:32,340 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:37,345 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:42,351 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:47:42,994 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:47:43,080 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:47:43,081 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "221.14.122.128","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "f01a31b9-5cc3-45f5-9736-ed1ece348cca","iid": "980d00ce-e7a6-486d-b464-5cf27a51b7d3","reference": "https://urlhaus.abuse.ch/url/324532/","rdata": "http://221.14.122.128:55591/mozi.m","last_at": "2020-03-20T19:02:31.979944Z","reported_at": "2020-03-20T19:02:31.979954Z"}
cif-router      | 2020-03-20 19:47:47,356 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:52,361 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:47:57,362 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:02,368 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:07,370 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:12,376 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:48:13,121 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:48:13,142 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:48:13,143 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "219.154.112.160","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "11f0ba4e-4058-4e9d-895e-8e7268798722","iid": "14e8cc45-72d3-4de7-95cc-d445c6bf3fdd","reference": "https://urlhaus.abuse.ch/url/324530/","rdata": "http://219.154.112.160:40597/mozi.m","last_at": "2020-03-20T19:02:31.981921Z","reported_at": "2020-03-20T19:02:31.981933Z"}
cif-router      | 2020-03-20 19:48:17,381 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:22,384 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:27,389 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:32,390 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:37,393 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:42,399 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:48:43,173 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:48:47,404 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:52,405 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:48:57,410 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:02,416 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:07,421 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:12,423 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:49:13,242 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:49:13,302 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:49:13,303 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "113.133.229.245","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "152ba529-50aa-4fda-bb79-41ec57127a77","iid": "e92ba704-796e-4e76-8cf3-23472673b38c","reference": "https://urlhaus.abuse.ch/url/324527/","rdata": "http://113.133.229.245:42749/mozi.m","last_at": "2020-03-20T19:02:31.984949Z","reported_at": "2020-03-20T19:02:31.984959Z"}
cif-router      | 2020-03-20 19:49:17,427 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:22,432 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:27,437 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:32,439 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:37,445 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:42,451 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:49:43,325 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:49:47,453 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:52,459 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:49:57,464 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:02,469 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:07,474 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:12,476 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:50:13,378 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:50:17,482 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:22,485 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:27,486 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:32,487 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:37,492 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:42,498 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:50:43,437 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:50:43,451 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in processcif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:50:43,451 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "110.155.76.52","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "eabd44b7-b2f7-4ad5-8f07-9632909d9d2b","iid": "65907425-beb2-4e7b-ae94-f5b2d3e90c54","reference": "https://urlhaus.abuse.ch/url/324523/","rdata": "http://110.155.76.52:37764/mozi.m","last_at": "2020-03-20T19:02:31.988768Z","reported_at": "2020-03-20T19:02:31.988778Z"}
cif-router      | 2020-03-20 19:50:47,501 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:52,505 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:50:57,510 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:02,515 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:07,516 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:12,520 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:51:13,521 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-router      | 2020-03-20 19:51:17,523 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:22,527 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:27,533 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:32,534 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:32,534 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:37,539 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-router      | 2020-03-20 19:51:42,545 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...
cif-hunter      | 2020-03-20 19:51:43,552 - ERROR - cif.hunter[113][MainThread] - EAGAIN: unable to create indicators.
cif-hunter      | 2020-03-20 19:51:43,575 - ERROR - cif.hunter[90][MainThread] - '127.0.0.11'
cif-hunter      | Traceback (most recent call last):
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in _process_plugin
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/cif-5.0b6-py3.7.egg/cif/hunter/__init__.py", line 83, in <listcomp>
cif-hunter      |     return [ii.__dict__() for ii in indicators if ii]
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py", line 12, in process
cif-hunter      |     i2 = i.spamhaus()
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_indicator/wrappers/spamhaus.py", line 12, in spamhaus
cif-hunter      |     rv = get(self.indicator)
cif-hunter      |   File "/usr/local/lib/python3.7/site-packages/csirtg_spamhaus/__init__.py", line 51, in get
cif-hunter      |     rv = IP_CODES[rv]
cif-hunter      | KeyError: '127.0.0.11'
cif-hunter      | 2020-03-20 19:51:43,575 - ERROR - cif.hunter[91][MainThread] - [<module 'spamhaus' from '/usr/local/lib/python3.7/site-packages/csirtg_hunter/plugins/spamhaus.py'>] giving up on {"indicator": "180.104.172.199","itype": "ipv4","tlp": "amber","provider": "urlhaus.abuse.ch","group": "everyone","count": 1,"tags": ["exploit","malware"],"confidence": 2.0,"description": "elf","uuid": "a5cb6730-eaa4-42a6-ae9e-573d86b47e3e","iid": "945e6687-b329-4e14-9441-89078949884a","reference": "https://urlhaus.abuse.ch/url/324520/","rdata": "http://180.104.172.199:34770/mozi.m","last_at": "2020-03-20T19:02:31.991637Z","reported_at": "2020-03-20T19:02:31.991647Z"}
cif-router      | 2020-03-20 19:51:47,551 - ERROR - cif.router.message[54][MainThread] - timeout sending to hunters...

Did you attempt to fix the problem and submit a pull request?

I searched through the codebase, looking through the search path, and I can't see any reason why it should be failing

Specifications like the version of the project, operating system, or hardware.

Ubuntu 18LTS with provided Vagrant file setup, latest version after your recent changes, 25gb storage, 2 cpu's, 8gb ram. (AWS t2.large)

How large is your /var/lib/cif.db database?

cifv5.db is 32M

ps. I apologize for spurring the need for an issue template. I will make sure I follow it from now on.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.