Comments (7)
Hi @SanWieb
Thank you for this input! We have something related to this on our backlog and I've incorporated your request to that. Right now it has a low priority given other items that we are working on.
Regards,
Ruben
from dettect.
I have the same problem. I haven't been able to get the YAML convert.
(DeTTECT-SwwsMdwy) arash@DESKTOP-IKKPGGL:~/DeTTECT$ python dettect.py ds -fd data-sources-new.yaml -l --health
[!] Data source: 'Third-party application logs' is MISSING from the YAML file
[!] Data source: 'Network device command history' is MISSING from the YAML file
[!] Data source: 'Network device run-time memory' is MISSING from the YAML file
[!] Data source: 'Network intrusion detection system' is MISSING from the YAML file
[!] Data source: 'OAuth audit logs' is MISSING from the YAML file
[!] Data source: 'API monitoring' is MISSING from the YAML file
[!] Data source: 'Binary file metadata' is MISSING from the YAML file
[!] Data source: 'PowerShell logs' is MISSING from the YAML file
[!] Data source: 'Process use of network' is MISSING from the YAML file
[!] Data source: 'Services' is MISSING from the YAML file
[!] Data source: 'Office 365 audit logs' is MISSING from the YAML file
[!] Data source: 'System calls' is MISSING from the YAML file
[!] Data source: 'Component firmware' is MISSING from the YAML file
[!] Data source: 'AWS CloudTrail logs' is MISSING from the YAML file
[!] Data source: 'Authentication logs' is MISSING from the YAML file
[!] Data source: 'Azure activity logs' is MISSING from the YAML file
[!] Data source: 'Process command-line parameters' is MISSING from the YAML file
[!] Data source: 'Loaded DLLs' is MISSING from the YAML file
[!] Data source: 'Social media monitoring' is MISSING from the YAML file
[!] Data source: 'WMI Objects' is MISSING from the YAML file
[!] Data source: 'Web proxy' is MISSING from the YAML file
[!] Data source: 'Netflow/Enclave netflow' is MISSING from the YAML file
[!] Data source: 'Process monitoring' is MISSING from the YAML file
[!] Data source: 'Email gateway' is MISSING from the YAML file
[!] Data source: 'BIOS' is MISSING from the YAML file
[!] Data source: 'Data loss prevention' is MISSING from the YAML file
[!] Data source: 'Windows Error Reporting' is MISSING from the YAML file
[!] Data source: 'Sensor health and status' is MISSING from the YAML file
[!] Data source: 'Domain registration' is MISSING from the YAML file
[!] Data source: 'AWS OS logs' is MISSING from the YAML file
[!] Data source: 'Access tokens' is MISSING from the YAML file
[!] Data source: 'EFI' is MISSING from the YAML file
[!] Data source: 'Web application firewall logs' is MISSING from the YAML file
[!] Data source: 'Application logs' is MISSING from the YAML file
[!] Data source: 'Named Pipes' is MISSING from the YAML file
[!] Data source: 'Anti-virus' is MISSING from the YAML file
[!] Data source: 'Detonation chamber' is MISSING from the YAML file
[!] Data source: 'Packet capture' is MISSING from the YAML file
[!] Data source: 'Digital certificate logs' is MISSING from the YAML file
[!] Data source: 'SSL/TLS certificates' is MISSING from the YAML file
[!] Data source: 'Malware reverse engineering' is MISSING from the YAML file
[!] Data source: 'Network device configuration' is MISSING from the YAML file
[!] Data source: 'VBR' is MISSING from the YAML file
[!] Data source: 'DLL monitoring' is MISSING from the YAML file
[!] Data source: 'Kernel drivers' is MISSING from the YAML file
[!] Data source: 'GCP audit logs' is MISSING from the YAML file
[!] Data source: 'Network protocol analysis' is MISSING from the YAML file
[!] Data source: 'SSL/TLS inspection' is MISSING from the YAML file
[!] Data source: 'Network device logs' is MISSING from the YAML file
[!] Data source: 'Asset management' is MISSING from the YAML file
[!] Data source: 'Windows Registry' is MISSING from the YAML file
[!] Data source: 'Office 365 account logs' is MISSING from the YAML file
[!] Data source: 'Web logs' is MISSING from the YAML file
[!] Data source: 'Azure OS logs' is MISSING from the YAML file
[!] Data source: 'MBR' is MISSING from the YAML file
[!] Data source: 'Host network interface' is MISSING from the YAML file
[!] Data source: 'Stackdriver logs' is MISSING from the YAML file
[!] Data source: 'Browser extensions' is MISSING from the YAML file
[!] Data source: 'Environment variable' is MISSING from the YAML file
[!] Data source: 'DNS records' is MISSING from the YAML file
[!] Data source: 'User interface' is MISSING from the YAML file
[!] Data source: 'File monitoring' is MISSING from the YAML file
[!] Data source: 'Office 365 trace logs' is MISSING from the YAML file
[!] Data source: 'Disk forensics' is MISSING from the YAML file
[!] Data source: 'Mail server' is MISSING from the YAML file
Traceback (most recent call last):
File "dettect.py", line 365, in <module>
_menu(_init_menu())
File "dettect.py", line 254, in _menu
generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, args.platform)
File "/home/arash/DeTTECT/data_source_mapping.py", line 24, in generate_data_sources_layer
my_techniques = _map_and_colorize_techniques(my_data_sources, platform, exceptions)
File "/home/arash/DeTTECT/data_source_mapping.py", line 289, in _map_and_colorize_techniques
determine_and_set_show_sub_techniques(output_techniques)
File "/home/arash/DeTTECT/generic.py", line 1166, in determine_and_set_show_sub_techniques
if len(subtech['techniqueID']) == 9:
TypeError: object of type 'NoneType' has no len()
Sample YAML file:
version: 1
file_type: data-source-administration
name: example
platform:
- all
data_sources:
- data_source_name: Web logs
date_registered: null
date_connected: null
products:
- Apache
available_for_data_analytics: false
comment: ''
data_quality:
device_completeness: 2
data_field_completeness: 2
timeliness: 2
consistency: 2
retention: 4
from dettect.
Hi @arashnikoo
Do you still have issues with that YAML? When using latest DeTT&CT version with all the packages from requirements.txt and using python 3.8, I don't get any error based on your given YAML file.
Regards,
Ruben
from dettect.
Hi @rubinatorz
I have the latest DeTT&CT, all the packages from requirements.txt are the latest, and using the python 3.8, still unable to convert the given YAML to file json.
Traceback (most recent call last):
File "dettect.py", line 365, in
_menu(_init_menu())
File "dettect.py", line 254, in _menu
generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, args.platform)
File "/root/DeTTECT/data_source_mapping.py", line 24, in generate_data_sources_layer
my_techniques = _map_and_colorize_techniques(my_data_sources, platform, exceptions)
File "/root/DeTTECT/data_source_mapping.py", line 261, in _map_and_colorize_techniques
total_ds_count = _count_applicable_data_sources(t, applicable_data_sources)
File "/root/DeTTECT/data_source_mapping.py", line 240, in _count_applicable_data_sources
ds = ds.split(':')[1][1:]
IndexError: list index out of range
Output for python3 dettect.py generic -ds
Count Data Source
243 Command Execution
197 Process Creation
95 File Modification
89 Network Traffic Content
84 Network Traffic Flow
82 File Creation
76 OS API Execution
58 Network Connection Creation
56 Windows Registry Key Modification
50 Application Log Content
49 Module Load
45 File Access
Traceback (most recent call last):
File "dettect.py", line 365, in
_menu(_init_menu())
File "dettect.py", line 321, in _menu
get_statistics_data_sources()
File "/root/DeTTECT/generic.py", line 1121, in get_statistics_data_sources
print(str_format.format(str(v['count']), k.split(':')[1][1:]))
IndexError: list index out of range
Could you please help?
from dettect.
Hi @cpaul82 it seems that both errors are the same. On both code lines the data source from the MITRE data is split based on the colon. In every data source there's a colon to split the data source and data component:
User Account: User Account Modification
We case the ATT&CK STIX data in the cache directory, and I think this local cache file is corrupt or outdated. Can you please try to remove the cache folder and try the command again?
from dettect.
hi @cpaul82
We've found the issue ("IndexError: list index out of range"), it is related to the newest version of the attackcti library. Please use version 0.3.3 as mentioned in the requirements.txt. See also my comment in issue #54.
from dettect.
Hi @rubinatorz I'm having a similar error:
" if len(subtech['techniqueID']) == 9:
TypeError: object of type 'NoneType' has no len()"
I've made sure the packages installed are the same versions listed in requirements.txt, as you mentioned above, but the error persists.
Any help would be appreciated
from dettect.
Related Issues (20)
- Feature Request: Export what's missing from Visibility or Detection HOT 3
- Receiving an error when converting data sources to json HOT 5
- Old versions of DeTTECT HOT 5
- unable to open jupyter notebook HOT 5
- error in connecting to MITRE's CTI TAXII server when convert yaml to json HOT 1
- Cannot connect to MITRE's CTI TAXII server HOT 1
- Unable to connect to Mitre Taxii Server HOT 2
- Feature request: add support for ATT&CK Version 14 HOT 2
- Some confusion about the "group" option HOT 1
- Duplicate value for applicable_to HOT 1
- arm64 docker image HOT 2
- Missing Data Sources By Technique
- Seeing some strangeness HOT 2
- Cannot connect to MITRE's CTI TAXII server HOT 2
- Current built mechnism broken - image likely outdated
- Latest techniques not appearing in editor HOT 1
- Feature request: Detection & Visibility overlay, highlighting where Visibility > Detection
- Question regarding detection rules/scoring HOT 1
- Question: How to handle non-mappable types of event? HOT 2
- Windows event log is not available while adding data source option. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dettect.