opencti-platform / connectors

OpenCTI Connectors

OpenCTI connectors

The following repository is used to store the OpenCTI connectors for the platform integration with other tools and applications. To know how to enable connectors on OpenCTI, please read the dedicated documentation.

Connectors list and statuses

This repository is used to host connectors that are supported by the core development team of OpenCTI. Nevertheless, the community is also developping a lot of connectors, third-parties modules directly linked to OpenCTI. You can find the list of all available connectors and plugins in the OpenCTI ecosystem dedicated space.

Contributing

If you want to help use improve or develop new connector, please check out the development documentation for new connectors. If you want to make your connector available to the community, please create a Pull Request on this repository, then we will integrate it to the CI and in the OpenCTI ecosystem.

License

Unless specified otherwise, connectors are released under the Apache 2.0. If a connector is released by its author under a different license, the subfolder corresponding to it will contain a LICENSE file.

About

OpenCTI is a product designed and developed by the company Filigran.

connectors's Issues

[ExportFileSTIX] Invalid STIX2 Export

Stix2 Validator is showing invalid, and is unable to import file into MISP.

[-] Results for: 2020-03-16T10_06_17.551Z_(ExportFileStix2)_report.json
[X] STIX JSON: Invalid
[!] Warning: marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9: {111} Open vocabulary value 'TLP' should be all lowercase and use hyphens instead of spaces or underscores as word separators.
[!] Warning: marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9: {201} Marking definition definition_type should be one of: statement, tlp.
[!] Warning: report--a12693d2-dc87-4e3c-a708-9da412d87185: {218} labels contains a value not in the report-label-ov vocabulary.
[X] report--a12693d2-dc87-4e3c-a708-9da412d87185: 'object_refs' is a required property
[X] observed-data--7112ab66-ad8c-48a2-b316-63a277d44189: 'first_observed' is a required property
[X] observed-data--7112ab66-ad8c-48a2-b316-63a277d44189: 'last_observed' is a required property
[X] observed-data--7112ab66-ad8c-48a2-b316-63a277d44189: 'created' is a required property
[X] observed-data--7112ab66-ad8c-48a2-b316-63a277d44189: 'modified' is a required property
[X] observed-data--7112ab66-ad8c-48a2-b316-63a277d44189: observed-data--7112ab66-ad8c-48a2-b316-63a277d44189: : Observed Data objects must be in dict format.

Basically the observed-data type is missing the 4 dates and I think the objects dictionary format should be like the below:

"objects": {
"0": {
"type": "x-new-observable",
"a_property": "foobaz",
"property_2": 5
}
}

Otherwise, thankyou for an interesting product. I look forward to each new release!

[ImportExternalUrl] Create the connector

Problem to Solve

There is currently no way to provide a URL to an Intel blog post or PDF report and have the IOCs parse our for ingestion into the platform. Intel analysts often have to manually extract out these IOCs when attempting to do analysis.

Current Workaround

You can manually specify a report and manually define IOCs (observables).

Proposed Solution

Provide the ability for a user to parse IOCs from an external report and ingest into the platform. The user would be able to specify the link to a public report and have the platform automatically parse out IOCs. The platform should do its best at identifying the Observable type parsed. The user should then have the ability to validate the parsed results, add context, label, etc. This could also be considered a bulk report IOC import capability.

Additional Information

None

CVE connector enhancement

Problem to Solve

• All CVE since 2003
• CVE JSON format import

Current Workaround

{ Please describe how you currently solve or work around this problem, given OpenCTI's limitation. }

Proposed Solution

{ Please describe the solution you would like OpenCTI to provide, to solve the problem above. }

Additional Information

{ Any additional information, including logs or screenshots if you have any. }

[TAXII 2] Create the connector

Problem to Solve

It would be nice to have a generic TAXII 2 Connector

Current Workaround

Manual JSON Import

Proposed Solution

Connector that implemented TAXII 2 protocol

MISP Connector (docker) fails with key error

Description

Running misp-connector in docker without providing the config as volumes. So just setting the right environment vars. MISP Connector fails caused with a key error wanting to get the values from the config.

Environment

1. OS (where OpenCTI server runs): Latest Docker
2. OpenCTI version: { e.g. OpenCTI 2.0.2 }
3. OpenCTI client: { e.g. frontend or python }
4. Other environment details:

Reproducible Steps

1. Create opencti-connector-misp container without configuration accessible through volumes and 2. login to the instance.
2. cd into /opt/opencti....
3. python ./misp.py

Expected Output

Running misp connector

Actual Output

Traceback (most recent call last):
  File "./misp.py", line 398, in <module>
    mispConnector = Misp()
  File "./misp.py", line 24, in __init__
    self.misp_tag = os.getenv('MISP_TAG') or config['misp']['tag'] if 'tag' in config['misp'] else None
KeyError: 'misp'

IPInfo connector failing at some IP's

Description

At some IP address enrichments the IPInfo connector is throwing an error

Environment

1. OS (where OpenCTI server runs): Debian
2. OpenCTI version: 2.0.2
3. OpenCTI client: frontend
4. Other environment details: -

Reproducible Steps

Add a new observable type IP4: 82.146.51.150 with IPInfo connector enabled. Notice that for this IP the connector is throwing an error

Expected Output

Enrichment of indicator

Actual Output

Fail in enrichment. Error thrown:

DEBUG:urllib3.connectionpool:https://ipinfo.io:443 "GET /82.146.51.150?token=########## HTTP/1.1" 200 None
ERROR:root:Error in message processing, reporting error to API
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/pycti/connector/opencti_connector_helper.py", line 43, in _data_handler
messages = self.callback(json_data)
File "ipinfo.py", line 66, in _process_message
bundle = self._generate_stix_bundle(country, json_data['city'], observable_id)
File "ipinfo.py", line 26, in _generate_stix_bundle
'x_opencti_alias': [country.official_name],
File "/usr/local/lib/python3.7/site-packages/pycountry/db.py", line 23, in getattr
raise AttributeError
AttributeError
INFO:root:Reporting job 3de7d75b-6574-4111-baf2-200e8e36aa09 with status error...

Additional information

{ Any additional information, including logs or screenshots if you have any. }

Fetch observables from local pdf reports

Problem to Solve

There is currently no way to automatically parse local pdf reports and import the resulting observables into OpenCTI

Current Workaround

None

Proposed Solution

Build a connector that would parse reports under a certain local directory and import the resulting observables into OpenCTI

Additional Information

Could easily be extended to csv/html reports

[New connector] COVID-19 - CyberThreatCoalition BlackList

Problem to Solve

Create a connector to be able to consume data from the official blacklist of the COVID-19 CyberThreat Coalition (https://www.cyberthreatcoalition.org/).

Current Workaround

Import lists with a custom Python script.

Proposed Solution

Create a full featured connector with:

• Import of indicators/observables
• Linked to ONE report named "COVID-19 CyberThreat Coalition BlackList".

Additional Information

Blacklist is here: https://blacklist.cyberthreatcoalition.org/

Reduce connectors docker image size

Problem to Solve

opencti/connectors docker image are big:

opencti/connector-cve                           1.1.2               507MB
opencti/connector-opencti                       1.1.2               507MB
opencti/connector-mitre                         1.1.2               507MB
opencti/connector-misp                          1.1.2               507MB

Proposed Solution

Clear cache, move to alpine base image.

[Cortex] Create the connector

Problem to Solve

Observables enrichment is currently not provided by OpenCTI. People wants to be able to display enrichment direclty in the platform.

Current Workaround

None.

Proposed Solution

Create the CORTEX connector for observables enrichment.

Additional Information

None.

Unable to import CVE feeds to Opencti by connector

Can you please provide me how to migrate CVE data of NVD feed using a connector.

Description

1. {1.Sucessfully add connector to docker-compose.yml.Though connector added unable to import data in opencti}

Environment

1. Ubuntu 18.04
2. OpenCTI 2.1.1.
3. OpenCTI client: { frontend }

Expected Output

{ CVE data into opencti }

Actual Output

{unable to migrate cve data to opencti,}

Integrate with multiple MISP Instances

Problem to Solve

As mentioned in the original issue, users would like to see the possibility to connect as many MISP instances as the admin desires. This could help people that have access to multiple of those instances.

Current Workaround

None.

Proposed Solution

Add the possibility to connect more than one instance

Additional Information

None.

Bug connector malpedia

Please replace every line in curly brackets { like this } with an appropriate answer, and remove this line.

Description

The connector malpedia don't work

Environment

1. OS (where OpenCTI server runs): CentOs 7
2. OpenCTI version: OpenCTI 3.1.0
3. OpenCTI client: Frontend
4. Other environment details:

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. cd /root
2. git clone https://github.com/OpenCTI-Platform/connectors.git
3. cd connectors/malpedia/
4. docker build -t connector-malpedia:3.1.0 .
5. I start my docker-compose with this connector

Actual Output

INFO:root:Listing Threat-Actors with filters null.
INFO:root:Starting ping alive thread
INFO:root:Fetching Malpedia datasets...
INFO:root:Connector has never run
INFO:root:Connector will run!
ERROR:root:unsupported operand type(s) for +: 'NoneType' and 'str'

[Intel Owl] Create the connector

Hi everyone,

thanks for this project and your contribution to the community.

I noticed that OpenCTI lacks of a chance to enrich observables.

I thought that a connector for the new Intel Owl project could be of interest: Intel Owl. In this way, OpenCTI users could leverage a single connector for the enrichment of observables or files.

Please let me know what do you think about.

IPinfo connector name is wrong

Description

Connector name of image is wrong
file : connectors/ipinfo/docker-compose.yml
line : 14

  connector-ipinfo:
    image: opencti/connector-import-file-stix:latest

MISP Connector doesn't support large volume

Description

The MISP connector know support to fetch every untagged events but doesnt provides any mechanism to control the events volume.

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Configure to get every untagged events in a large MISP
2. Start the connector

Actual Output

Timeout on the MISP query

Expected Output

Limit the volume to get incrementally all the MISP events

Proposed solution

Add a new option in the config to limit the numbers of MISP elements fetched in one query. See limit in https://pymisp.readthedocs.io/_modules/pymisp/aping.html.
Setup this number to 100 by default.

Is there a URI for the OpenCTI API for testing?

Please replace every line in curly brackets { like this } with an appropriate answer, and remove this line.

Description

All of the connectors from this repot (other than the import and export) fail to connect to the API but other services are running

Environment

1. OS: Ubuntu 18:04
2. OpenCTI version: 3.1.0
3. OpenCTI client: frontemd

Reproducible Steps

Steps to create the smallest reproducible scenario:

Added the connector container to the docker-compose.yml file and restart the stack. The connectors are active and OpenCTI appears to be working but none of the connectors can contact the API.

Expected Output

I expect that the connectors to MISP/VT/COVID would work.

Actual Output

OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...

Additional information

None

alienvault connector : always insert URL of the pulse as an external reference

Please replace every line in curly brackets { like this } with appropriate answers, and remove this line.

Problem to Solve

When importing a pulse in the form of a report, the alienvault connector creates an external reference when the orginal alienvault pulse provides an URL under in the field REFERENCE.
However, some alienvault pulses do not populate this REFERENCE field. In this case, no external reference is created in the opencti report and it is impossible, from OpenCTI, to find the origin of the information.

Current Workaround

{ Please describe how you currently solve or work around this problem, given OpenCTI's limitation. }

Proposed Solution

When importing a pulse from alienvault, the connector should always add the URL of the alienvault as an external reference
(in addition to any existing URL provided under REFERENCE in the pulse).

Additional Information

{ Any additional information, including logs or screenshots if you have any. }

CrowdStrike connector

Problem to Solve

Get actors, reports and indicators from CrowdStrike.

[TheHive] Create the connector

Problem to Solve

Find a way to synchronize TheHive cases (and associated observables) to OpenCTI.

Current Workaround

None.

Proposed Solution

Create a bi-directionnal TheHive connector.

Additional Information

None.

Fix antlr version

I am currently getting the following error with this connector:

ANTLR runtime and generated code versions disagree: 4.7.2!=4.8

Could it be, that the hard coded 4.7.2 version needs to be removed @maertv ?

connectors/alienvault/Dockerfile

Line 12 in 5d86c52

pip install --no-cache-dir antlr4-python3-runtime==4.7.2 && \

[VirusTotal Hunting] Create the connector

To obtain IOCs (hashes values, ...) and associated context (submit date, country, etc) for surveilled malware families.

Yara rules are provided as an entry by the susbriber

https://www.virustotal.com/gui/hunting-overview

MISP integration - events\IOCs from MISP not shown on the OpenCTI dashboard

Hi,
After the initial settings of MISP integration no data shown in the OpenCTI dashboard.
I can't see any ingested data from MISP. The only thing that works correctly is the imported_tag: 'OpenCTI: Imported'. but the IOCs of the MISP events not shown.

Reproducible Steps

• I validate that the misp.py is run without any issues directly from the CLI.
• Network connection from OpenCTI to MISP in port 443 is alright.
• config.yml settings:
  rabbitmq:
  hostname: 'localhost'
  port: 5672
  username: 'guest'
  password: 'guest'

misp:
name: 'MISP' # Required
confidence_level: 3 # Required
url: 'https://misp.test.local' # Required
key: 'xxxxxxxxxxxxxxxxxxxxxxxx' # Required
tag: 'attack2' # Optional, tags of events to be ingested (if not provided, import all!)
untag_event: False # Optional, remove the tag after import
imported_tag: 'OpenCTI: Imported' # Required, tag event after import
filter_on_imported_tag: True # Required, use imported tag to know which events to not ingest
interval: 1 # Minutes
log_level: 'info'

Environment

OS Ubuntu 18.04
OpenCTI version: Version 1.1.2
Manual installation.

Create the OpenCTI connector for default datasets

Problem to Solve

For default datasets such as sectors, countries, regions and cities. An OpenCTI connector can be a good solution to maintain up-to-date entities.

Current Workaround

Create those entities manually.

Proposed Solution

Create an OpenCTI connector and the associated datasets.

Additional Information

None.

Adding a state indicator to enable incremental updates

Problem to Solve

Currently the import process does not foresee in duplicate entities being processed during import. Every time when for instance the Mitre connector is started the complete import process is starting for scratch. It would be good to have only incremental updates being processed

Current Workaround

Add more workers to speed up the import process or extend the interval period. This does only speed up the process

Proposed Solution

Adding a state indicator to prevent entities being processed that have not been changed since last import.

Additional Information

None

Make the MISP connector bi-directional

Problem to Solve

People wants to be able to push events/attributes to MISP from OpenCTI.

Current Workaround

None.

Proposed Solution

Make the MISP connector to be bi-directional.

Additional Information

None.

[ImportExternalUrl] Create the connector

Import public threat reports from open sources blogs.

• The html page should be transformed in to an OpenCTI report
• The URL of the page should be transformed into an external reference
• If the page contains tags (i.e. related to malware families, threat actors, countries, intrusion sets) automatic associations should be made in OpenCTI

For each blog, there could be two different work flows:

1. Select mode = Select one by one the blog posts to be imported, from any source (page as report and url as a external reference)

OR

2. Subscription mode = Get automatically all posts from a specific blog (page as report and url as external reference). Validate or validate the posts that the analyst want to keep.

Examples of open sources blogs :

IT security news aggregators such as:

• Security affairs
• Bleeping computer
• ZDNet
• ThreatPost
• etc

IT security vendor owned blogs:

• Kaspersky securelist
• ESET
• Trendlabs
• etc

Refactor and enhance the MISP connector

Problem to Solve

The MISP connector is currently working but difficult to understand and use.

Current Workaround

None.

Proposed Solution

Completely refactor the connector to make it more useful/efficient with understandable and flexible parameters.

Additional Information

None.

Alienvault OTX connector

Collect IOCs from AlienvaultOTX

Create automatically associated reports
Create automatically associated links

Use Alienvault tags to create automatically

• associated threats (threat actors, intrusion sets, malware)
• assocated countries

https://www.alienvault.com/documentation/otx.htm?tocpath=Documentation%7CAlienVault%C2%AE%20Open%C2%A0Threat%20Exchange%C2%AE%7C_____0

OpenCTI couldn't get a MISP data

Description

When running the MISP connector, OpenCTI couldn't get a MISP data.

Environment

1. OS (where OpenCTI server runs): ubuntu 19.10 VirtualBox
2. OpenCTI version : 3.0.2
3. OpenCTI client : python 3.7.5
4. Other environment details: pymisp==2.4.119.1

Reproducible Steps

Clone the repository

$ mkdir /path/to/your/app && cd /path/to/your/app
$ git clone https://github.com/OpenCTI-Platform/docker.git
$ cd docker

misp connector download

Docker Set configure

• APP__ADMIN__PASSWORD
• APP__ADMIN__TOKEN
• OPENCTI_TOKEN
• vm.max_map_count

MISP configure

• opencti-url
• opencti-token
• misp-url
• misp-key
• misp-ssl_verify

Run

$ docker-compose --compatibility up
$ python3 misp.py

Expected Output

I wanted to importing MISP data inside openCTI.

Actual Output

MISP returned 0 events

$ python3 misp.py
INFO:root:Connector last run: 2020-02-26 07:02:25
INFO:root:Fetching MISP events with args: {"tags": {"OR": ["opencti:import", "type:osint"]}, "timestamp": "2020-02-26 07:02:25", "limit": 100, "page": 1}
INFO:root:MISP returned 0 events.

Additional information

[AlienVault] Report author = alienvault pulse user

Please replace every line in curly brackets { like this } with appropriate answers, and remove this line.

Problem to Solve

Currently, all reports imported wth alienvault connector have author = Alienvault.
However, Alienvault is a community platform where several users shares reports and indicators.
So it is important to know which user s the author of the report (pulse).

Current Workaround

{ Please describe how you currently solve or work around this problem, given OpenCTI's limitation. }

Proposed Solution

Set report author = alienvault pulse user

Note: the information that the report comes from the alienvault platform should be recorded thanks to issue OpenCTI-Platform/opencti#566 (comment)

Additional Information

{ Any additional information, including logs or screenshots if you have any. }

MISP connector does not import some types

Afffected types:

• YARA rules
• Emails subject/address

Full refactor of connectors

Problem to Solve

Following the platform architecture issue, connectors must be refactored to integrate with this new architecture.

Current Workaround

None.

Proposed Solution

Each connector will be launched in a specific container and will be independant from the rest of the platform.

Implement, for each connector:

• a built-in scheduler capability provided by OpenCTI (developers will only focus on the connector)
• be sure that each connector produce an format of data handled by the importation workers (currently only STIX2)
• integrate a docker-compose in each connector
• push default connectors in the global docker-compose.yml of the platform.

A CircleCI configuration will generate a Docker image for each connector.

Additional Information

None.

Alienvault and Crowdstrike connectors - use author's tag to indicate malware

Please replace every line in curly brackets { like this } with appropriate answers, and remove this line.

Problem to Solve

If the author uses a simple tag to refer to a malware, this information will be imported as a simple tag and not included as "knowledge". Furthermore, there will be no "indicate" relationship between the indicators imported with the report and the malware family.

Current Workaround

{ Please describe how you currently solve or work around this problem, given OpenCTI's limitation. }

Proposed Solution

Use tags provided by the author of the report and check if they match a malware existing in opencti.
In such case, include the malware in the knowledge of the imported report and create an "indicate" relationship between the imported indicators and the malware.

Additional Information

{ Any additional information, including logs or screenshots if you have any. }

Upgrade all connectors with the new helper and configuration

Problem to Solve

The current connectors are not compatible with the new version of the platform.

Current Workaround

None.

Proposed Solution

Upgrade all connectors.

Additional Information

None.

CrowdStrike indicator import fails because of OpenCTI observable type mismatch

Description

CrowdStrike indicator import fails because of OpenCTI observable type mismatch.

Expected Output

CrowdStrike indicator is imported to OpenCTI.

Actual Output

CrowdStrike indicator import fails.

Sample OpenCTI error message:

"Error: [SCHEMA] Observable type url is not supported."

RabbitMQ connexion doesn't work (example MISP)

Description

When I configure OpenCTI and wanting to add some connector, I've configured my connector, but they crash because the reach OpenCTI API, but cannot use correctly AQMP.

Environment

1. Linux ubuntu-virtual-machine 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux (with uname -a)
2. OpenCTI version: { e.g. OpenCTI 1.0.2 }
3. OpenCTI client: frontend

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Install OpenCTI latest with docker
2. Configure it to a local use (localhost)
3. git clone https://github.com/OpenCTI-Platform/connectors
4. cd connectors/misp/src
5. sudo pip3 install -r requirements.txt
6. Configuring config.yml :

  url: 'http://127.0.0.1:8080'
  token: 'myToken'

connector:
  id: 'myToken'
  type: 'EXTERNAL_IMPORT'
  name: 'MITRE ATT&CK'
  scope: 'identity,attack-pattern,course-of-action,intrusion-set,malware,tool,report'
  confidence_level: 3
  update_existing_data: True
  log_level: 'info'

mitre:
  enterprise_file_url: 'https://raw.githubusercontent.com/mitre/cti/master/enterprise-attack/enterprise-attack.json'
  pre_attack_file_url: 'https://raw.githubusercontent.com/mitre/cti/master/pre-attack/pre-attack.json'
  interval: 7 # Days

Expected Output

Just wanting to see importation of files.

Actual Output

$ python3 mitre.py 
INFO:root:Listing Threat-Actors with filters null.
INFO:root:Starting ping alive thread
test
INFO:root:Fetching MITRE datasets...
INFO:root:Connector has never run
INFO:root:Connector will run!
ERROR:pika.adapters.utils.selector_ioloop_adapter:Address resolution failed: gaierror(-2, 'Name or service not known')
ERROR:pika.adapters.utils.connection_workflow:getaddrinfo failed: gaierror(-2, 'Name or service not known').
ERROR:pika.adapters.utils.connection_workflow:AMQP connection workflow failed: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - gaierror(-2, 'Name or service not known'); first exception - None.
ERROR:pika.adapters.utils.connection_workflow:AMQPConnectionWorkflow - reporting failure: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - gaierror(-2, 'Name or service not known'); first exception - None
ERROR:pika.adapters.blocking_connection:Connection workflow failed: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - gaierror(-2, 'Name or service not known'); first exception - None
ERROR:pika.adapters.blocking_connection:Error in _create_connection().
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pika/adapters/blocking_connection.py", line 450, in _create_connection
    raise self._reap_last_connection_workflow_error(error)
  File "/usr/local/lib/python3.6/dist-packages/pika/adapters/utils/selector_ioloop_adapter.py", line 564, in _resolve
    self._flags)
  File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

Additional information

When I see log of OpenCTI, I see :

rabbitmq_1                               | 2020-03-05 13:04:01.574 [info] <0.16172.0> connection <0.16172.0> (172.19.0.13:52248 -> 172.19.0.2:5672): user 'guest' authenticated and granted access to vhost '/'
rabbitmq_1                               | 2020-03-05 13:04:01.589 [info] <0.16172.0> closing AMQP connection <0.16172.0> (172.19.0.13:52248 -> 172.19.0.2:5672, vhost: '/', user: 'guest')

I gess the script can access to OpenCTI API, and to RabbitMQ, but crash after, maybe misconfiguration of pika ?

Remove antlr4-python3-runtime==4.7.2 from CrowdStrike connector Dockerfile

It is not required since https://github.com/oasis-open/cti-pattern-validator/releases/tag/v1.3.0 release.

It also causes following errors:

ANTLR runtime and generated code versions disagree: 4.7.2!=4.8

Unable to connect to MISP

Description

In connector-misp_1, we receive the following error:

"connector-misp_1 | Request body: connector-misp_1 | {"returnFormat": "json", "tags": {"AND": ["OpenCTI:\ Import"], "NOT": ["OpenCTI:\ Imported"]}, "withAttachments": 0, "metadata": 0, "enforceWarninglist": 0, "includeEventUuid": 0, "sgReferenceOnly": 0, "includeContext": 0, "headerless": 0, "includeSightings": 0, "includeCorrelations": 0}
connector-misp_1 | Response (if any):
connector-misp_1 | {"name":"An Internal Error Has Occurred.","message":"An Internal Error Has Occurred.","url":"/events/restSearch"}
connector-misp_1 | CRITICAL:pymisp:Unknown error: the response is not in JSON.
connector-misp_1 | Something is broken server-side, please send us everything that follows (careful with the auth key):
connector-misp_1 | Request headers:
connector-misp_1 | {'User-Agent': 'PyMISP 2.4.112 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '290', 'Authorization': 'hXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', 'content-type': 'application/json'}
."

Environment

1. OS: Centos 7
2. OpenCTI version:OpenCTI 1.1.2
3. OpenCTI client: frontend or python }
4. Other environment details

Expected Output

Data from MISP showing on OpenCTI interface

Actual Output

No data from MISP to OpenCTI platforms

[MITRE] Beta version of MITRE ATT&CK ® Matrix with sub-techniques

Problem to Solve

Beta version of MITRE ATT&CK ® Matrix with sub-techniques is not available in OpenCTI.

Proposed Solution

Create a new connector with the beta version of the matrix.

OR

Update the mitre connector with a BETA flag to easily switch from stable to beta version of the matrix.

Additional Information

• ATT&CK with Sub-Techniques — What You Need to Know
  • "With sub-techniques published in beta, the next two efforts we will focus on from our 2020 roadmap are revamping the Data Sources used by Enterprise and taking the next step to merge PRE-ATT&CK into ATT&CK."
• MITRE ATT&CK ®: Design and Philosophy

MISP Connector "InsecureRequestWarning"

I am working as a summer intern, so I am new to both MISP and OpenCTI

Description

When running docker-compose up after adding MISP connector to the docker-compose.yml an InsecureRequestWarning error is shown.

Environment

1. OS (where OpenCTI server runs): Ubuntu 18.04 VM hosted on AWS
2. OpenCTI version: e.g. OpenCTI 1.1.1
3. OpenCTI client: frontend

Actual Output

`connector-misp_1 | /usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

connector-misp_1 | InsecureRequestWarning)`

Additional information

This source was modified and added to the docker-compose.yml to allow the MISP connector. Without the MISP portion, docker-compose up works correctly.

Automatic Ingest of Threat Feeds

It would be good to be able to ingest opensource and commercial threat feeds automatically by choosing a feed url.

Current Workaround

Indicators need to be imported manually.

Proposed Solution

• Have the ability to select from a range of opensource feeds and have them automatically ingest either on demand, or once per day etc.
• Be able to categorise feeds being ingested into appropriate categories (i.e. Phishing IP's; C2 Domains; Hashes of Malware etc)

Additional Information

[CVE] Download link to variable

Description

Set the download CVE link to variable, because otherwise the tool can hardly be used offline. Offline we can host the CVEs on a link that is not : "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-"

Reproducible Steps

https://github.com/OpenCTI-Platform/connectors/blame/9d47ffdad1c2a7fbdd709565d5c3f670693b148f/cve/src/cve.py#L103

Expected Output

Url as a variable in the .yml

Actual Output

Permanent link : "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-"

[ImportFileCsv] Create the connector

Problem to Solve

Users should be able to import data from CSV files. Create a connector on the model "ImportFileStix".

Current Workaround

None.

Proposed Solution

Create the CSV connector as well as the documentation about the expected CSV columns.

Additional Information

None.

[AlienVault] Importing old pulses errors

Description

When trying to import old pulses (pulse_start_timestamp: '2018-01-01T00:00:00' # ISO 8601), the connector throw errors.

The errors are:

INFO:root:Running pulse importer (update data: False, guess malware: False)...
ERROR:root:23 validation errors for ParsingModel[List[alienvault.models.Pulse]]
__root__ -> 320 -> adversary
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 67 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 68 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 69 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 70 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 71 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 72 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 73 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 74 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 75 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 76 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 77 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 78 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 79 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 80 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 81 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 82 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 527 -> indicators -> 83 -> content
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 684 -> adversary
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 726 -> adversary
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 742 -> adversary
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 1001 -> adversary
  none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> 1008 -> adversary
  none is not an allowed value (type=type_error.none.not_allowed)

Environment

AlienVault Connector 3.1.0

Parallel ingestion of data

Problem to Solve

Currently, data from connectors is sequentially ingested. If there are a lot of entities/observables to ingest, it could take some time depending of the server hardware performances where Grakn is deployed.

Current Workaround

None.

Proposed Solution

Refactor the connectors to user the import workers.

Additional Information

None.

[Malpedia] Create the connector

Please replace every line in curly brackets { like this } with appropriate answers, and remove this line.

Problem to Solve

{ Please describe the problem you would like to solve. }

Current Workaround

{ Please describe how you currently solve or work around this problem, given OpenCTI's limitation. }

Proposed Solution

{ Please describe the solution you would like OpenCTI to provide, to solve the problem above. }

Additional Information

{ Any additional information, including logs or screenshots if you have any. }

[CrowdStrike] Import yara rules master

Please replace every line in curly brackets { like this } with appropriate answers, and remove this line.

Problem to Solve

Yara rules master is not imported.

Current Workaround

{ Please describe how you currently solve or work around this problem, given OpenCTI's limitation. }

Proposed Solution

The weekly yara rules master shall be imported.

Replace existing rules with rules having the same name, according the the following logic

• Crowdstrike yara rules contain a metadata field "last_modified" which is always a Friday
• Crowdstrike yara rules are grouped in master package available every Friday
• So, in the import process, a rule having a name which already exists should replace the existing rule only if last_modified = current import date

Associate each yara rule to relevant intrusion sets or malware family based on meta data:

• actor = "PRIMITIVE BEAR" → intrusion set = "PRIMITIVE BEAR"
• malware_family = "Triceratops" → malware = "Triceratops"

Additional Information

{ Any additional information, including logs or screenshots if you have any. }

ImportfilePDFobservables connector --> PDF not imported as a report and obsrvables not attached to it

Please replace every line in curly brackets { like this } with an appropriate answer, and remove this line.

Description

I tested the connector with PDF file.
Observables seem to be correctly extracted but

• the PDF document is not imported as report
  and, consequently
• the observables are not attached to a report
• it is not possible to easily find them in the general set of observables
• it is not possible to remove false positive (e.g. the PDF report include some domain names or email addresses that should NOT be observables. For example the contact email of the organisation that created the PDF, etc)

Environment

1. OS (where OpenCTI server runs): { e.g. Mac OS 10, Windows 10, Ubuntu 16.4, etc. }
2. OpenCTI version: { e.g. OpenCTI 1.0.2 }
3. OpenCTI client: { e.g. frontend or python }
4. Other environment details:

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. { e.g. Run ... }
2. { e.g. Click ... }
3. { e.g. Error ... }

Expected Output

{ Please describe what you expected to happen. }

Actual Output

{ Please describe what actually happened. }

Additional information

{ Any additional information, including logs or screenshots if you have any. }

cve and ipinfo connectors - ERRR:root:You are not allowed to do this.

Hi,

everything else is up and running fine however cvs and ipinfo connectors seem not to be working. Configs were copied over from github and added to docker-compose.yml as below.

docker-compose.yml
....
connector-ipinfo:
image: opencti/connector-ipinfo:latest
environment:
- OPENCTI_URL=http://opencti:8080
- OPENCTI_TOKEN=xxxxxxxxxxxxxxxx
- CONNECTOR_ID=xxxxxxxxxxxxxxxxx
- CONNECTOR_TYPE=INTERNAL_ENRICHMENT
- CONNECTOR_NAME=IpInfo
- CONNECTOR_SCOPE='ipv4-addr'
- CONNECTOR_CONFIDENCE_LEVEL=3
- CONNECTOR_LOG_LEVEL=info
- IPINFO_TOKEN=XXXXXXXXXXXXX
- HTTP_PROXY=http://x.x.x.x:3128/
- HTTPS_PROXY=http://x.x.x.x:3128/
- NO_PROXY=localhost,opencti,grakn,127.0.0.1
restart: always
connector-cve:
image: opencti/connector-cve:latest
environment:
- OPENCTI_URL=http://opencti:8080
- OPENCTI_TOKEN=xxxxxxxxxxxxxxxx
- CONNECTOR_ID=xxxxxxxxxxxxxxxx
- CONNECTOR_TYPE=EXTERNAL_IMPORT
- CONNECTOR_NAME=Common Vulnerabilities and Exposures
- CONNECTOR_SCOPE=identity,vulnerability
- CONNECTOR_CONFIDENCE_LEVEL=3
- CONNECTOR_UPDATE_EXISTING_DATA=true
- CONNECTOR_LOG_LEVEL=info
- CVE_NVD_DATA_FEED=https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-recent.json.gz
- CVE_INTERVAL=1 # Days
- HTTP_PROXY=http://x.x.x.x:3128/
- HTTPS_PROXY=http://x.x.x.x:3128/
- NO_PROXY=localhost,opencti,grakn,127.0.0.1
restart: always

Any idea?
thx