f5-labs / cryptonice Goto Github PK

CryptoNice is both a command line tool and library which provides the ability to scan and report on the configuration of SSL/TLS for your internet or internal facing web services. Built using the sslyze API and ssl, http-client and dns libraries, cryptonice collects data on a given domain and performs a series of tests to check TLS configuration and supporting protocols such as HTTP2 and DNS.

Home Page: https://www.f5.com/labs/cryptonice

License: GNU General Public License v3.0

Python 99.86% Dockerfile 0.14%

appsec tls security security-tools crypto ssl ssl-certificates tls-scan tls-certificate dns

cryptonice's People

Contributors

Stargazers

Watchers

cryptonice's Issues

AttributeError: 'PosixPath' object has no attribute 'read_text'

When trying to install it with pip I get:
root@host:~# pip install cryptonice
Collecting cryptonice
Using cached https://files.pythonhosted.org/packages/c4/eb/5d5df1a5ae9056a90bf5e620684924c7971dccf037e3d2b3578659540f58/cryptonice-0.1.17.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-noio1d/cryptonice/setup.py", line 11, in
README = (HERE / "README.md").read_text()
AttributeError: 'PosixPath' object has no attribute 'read_text'

----------------------------------------

Console output is missing Supported Protocols

Since 2020-07-09 the console no longer displays the Supported Protocols after a TLS scan

RESULTS
--------------------
Hostname:                         accounts.google.com

Selected Cipher Suite:            TLS_AES_256_GCM_SHA384
Selected TLS Version:             TLS_1_3

Supported protocols:

HTTP/2 supported:                 True

cryptonice should create multiple output files if more than one target is specified

Current behaviour:
Multiple targets specified in the job file will result in only the last target being output to JSON with the filename also of the last target.

Expected behaviour:
Specifying multiple targets in the input file will result in a separate output JSON being created for each one.

revoked.badssl.com and pinning-test.badssl.com have trusted certificates

The two mentioned sites don't cause any errors to be generated (in cryptonice nor in sslyze). Not sure what the issue here is, so I'm not sure how to catch them.

Some site redirection methods produce errors in gethttp.py

e.g. latimes.com

Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 55, in lambda_handler
    output_data, hostname = scanner.scanner_driver(input_data)
  File "/opt/python/lib/python3.7/site-packages/cryptonice-1.3.7.1-py3.7.egg/cryptonice/scanner.py", line 414, in scanner_driver
    force_redirect)
  File "/opt/python/lib/python3.7/site-packages/cryptonice-1.3.7.1-py3.7.egg/cryptonice/gethttp.py", line 124, in get_http
    conn.request("GET", str_path)
  File "/var/lang/lib/python3.7/http/client.py", line 1277, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/var/lang/lib/python3.7/http/client.py", line 1323, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/var/lang/lib/python3.7/http/client.py", line 1272, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/var/lang/lib/python3.7/http/client.py", line 1032, in _send_output
    self.send(msg)
  File "/var/lang/lib/python3.7/http/client.py", line 972, in send
    self.connect()
  File "/var/lang/lib/python3.7/http/client.py", line 1439, in connect
    super().connect()
  File "/var/lang/lib/python3.7/http/client.py", line 944, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/var/lang/lib/python3.7/socket.py", line 707, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/var/lang/lib/python3.7/socket.py", line 752, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

This one is interesting. If you manually go to latimes.com using openssl s_client, you can see this:

GET / HTTP/1.0

HTTP/1.1 301 Moved Permanently
Server: awselb/2.0
Date: Thu, 01 Jul 2021 20:15:23 GMT
Content-Type: text/html
Content-Length: 134
Connection: close
Location: https://www.internal-caltimes-root-redirector-alb-1421749447.us-west-2.elb.amazonaws.com:443/

But following the Location link, you get this:

openssl s_client -connect www.internal-caltimes-root-redirector-alb-1421749447.us-west-2.elb.amazonaws.com:443
getaddrinfo: Name or service not known
connect:errno=2

Thus the error. Attempting to connect to the Location uri using curl gives the same error.

In a browser, however, it works fine.

AttributeError: 'SessionRenegotiationScanResult' object has no attribute 'accepts_client_renegotiation'

Python 3.9.2
Cryptonice 1.3.4 installed from pip

# cryptonice networks.h-net.org --tls all
Pre-scan checks
-------------------------------------
Scanning networks.h-net.org on port 443...
Analyzing DNS data for networks.h-net.org
Fetching additional records for networks.h-net.org
networks.h-net.org resolves to 35.9.18.76
35.9.18.76:443: OPEN
TLS is available: True
Connecting to port 443 using HTTPS
Reading HTTP headers for networks.h-net.org
Queueing TLS scans (this might take a little while...)
Traceback (most recent call last):
  File "/usr/local/bin/cryptonice", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/cryptonice/__main__.py", line 104, in main
    output_data, hostname = scanner_driver(input_data)
  File "/usr/local/lib/python3.9/dist-packages/cryptonice/scanner.py", line 437, in scanner_driver
    tls_data = tls_scan(ip_address, str_host, commands_to_run, port)
  File "/usr/local/lib/python3.9/dist-packages/cryptonice/gettls.py", line 551, in tls_scan
    {'accepts_client_renegotiation': renegotiation_results.accepts_client_renegotiation})
AttributeError: 'SessionRenegotiationScanResult' object has no attribute 'accepts_client_renegotiation'

JSON formatting issues

Bringing JSON results in to ELK 7.9 is problematic due to JSON formatting.

tls.tests.vulnerable_to_robot returns an array containing a boolean and then a string. These should be separated out to two different fields.
http.Cookies contains an empty string if no results are available. This should be null

HTTP to HTTPS redirects not always correctly identified

A number of sites redirect from HTTP to HTTPS but this is not detected within cryptonice.

Name or service not known (from docker)

against echo.epa.gov

socket.gaierror: [Errno -2] Name or service not known

DNS resolution issues

We’re checking to see if the given value is an IP or a name, and since it’s a name passing it to the get_dns function to look it up.
This returns, but it returns an empty value in a list, because the name is not resolvable (dns_data.get('records').get('A')[0] # get first IP in list) and so we get the index error.

We should add a check somewhere to handle cases where the DNS name handed in is not resolvable.

===============

facebook.net

Traceback (most recent call last):
  File "/opt/python/lib/python3.7/site-packages/cryptonice-1.3.7.1-py3.7.egg/cryptonice/scanner.py", line 374, in scanner_driver
    ipaddress.ip_address(hostname)
  File "/var/lang/lib/python3.7/ipaddress.py", line 54, in ip_address
    address)
ValueError: 'facebook.net' does not appear to be an IPv4 or IPv6 address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "/opt/python/lib/python3.7/site-packages/cryptonice-1.3.7.1-py3.7.egg/cryptonice/scanner.py", line 386, in scanner_driver
    ip_address = dns_data.get('records').get('A')[0]  # get first IP in list
IndexError: list index out of range

Update Wappalyzer component to 6.7.7

Support for test output formats like JUnit

Hello,

I use cryptonice often to verify if my internet facing endpoints are configured properly. This configuration is done in K8s or on a cloud platform such as Azure with Azure Resource Manager templates. As soon as I test changes I need to re-run cryptonice to verify the changes. I can do this in a pipeline as a post-deployment step, but I have to go through the agent logs to check the cryptonice output. To simplify this step, it would be handy if cryptonice supports output formats such as JUnit, which can be visualised by all common CI/CD solutions.

Please let me know what you think about this enhancement.

Thank you and regards
Marc

Some sites not returning TLS/certificate info despite TLS being available

For example buzzfeed.com

cryptonice buzzfeed.com --scans tls dns http jarm --tls all

Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 55, in lambda_handler
    output_data, hostname = scanner.scanner_driver(input_data)
  File "/opt/python/lib/python3.7/site-packages/cryptonice-1.3.7.1-py3.7.egg/cryptonice/scanner.py", line 453, in scanner_driver
    cert_fingerprint = tls_data['certificate_info']['certificate_0']['fingerprint']
KeyError: 'certificate_info'

This one is simply that the certificate_info key isn’t set.
That’s likely due to tls_data = tls_scan(ip_address, str_host, commands_to_run, port) not always setting it.

AttributeError: 'CipherSuitesScanResult' object has no attribute 'cipher_suite_preferred_by_server'

$ cryptonice https://gogle.com
Pre-scan checks
-------------------------------------
Scanning gogle.com on port 443...
Analyzing DNS data for gogle.com
Fetching additional records for gogle.com
gogle.com resolves to 142.250.74.196
142.250.74.196:443: OPEN
TLS is available: True
Connecting to port 443 using HTTPS
Reading HTTP headers for www.google.com
Queueing TLS scans (this might take a little while...)
0
1
Traceback (most recent call last):
  File "/home/math/.pyenv/versions/3.8.6/bin/cryptonice", line 8, in <module>
    sys.exit(main())
  File "/home/math/.pyenv/versions/3.8.6/lib/python3.8/site-packages/cryptonice/__main__.py", line 104, in main
    output_data, hostname = scanner_driver(input_data)
  File "/home/math/.pyenv/versions/3.8.6/lib/python3.8/site-packages/cryptonice/scanner.py", line 427, in scanner_driver
    tls_data = tls_scan(ip_address, str_host, commands_to_run, port)
  File "/home/math/.pyenv/versions/3.8.6/lib/python3.8/site-packages/cryptonice/modules/gettls.py", line 273, in tls_scan
    preferred_cipher_suite = ssl2_result.cipher_suite_preferred_by_server
AttributeError: 'CipherSuitesScanResult' object has no attribute 'cipher_suite_preferred_by_server'

System information

$ python --version
Python 3.8.6

$cryptonice -v
1.2.5

error in Docker when scanning localhost 'UnboundLocalError: local variable 'jarm_data' referenced before assignment'

how to reproduce - just run with no parameters on a machine with nothing listening on port 443

Scanning localhost on port 443...
Analyzing DNS data for localhost
Fetching additional records for localhost
localhost resolves to 127.0.0.1
localhost:443 is closed
Traceback (most recent call last):
  File "/usr/local/bin/cryptonice", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/cryptonice/__main__.py", line 104, in main
    output_data, hostname = scanner_driver(input_data)
  File "/usr/local/lib/python3.7/site-packages/cryptonice/scanner.py", line 459, in scanner_driver
    if jarm_data:
UnboundLocalError: local variable 'jarm_data' referenced before assignment

AttributeError: 'CipherSuitesScanResult'

$cryptonice f5.com
Pre-scan checks

Scanning f5.com on port 443...
Analyzing DNS data for f5.com
Fetching additional records for f5.com
f5.com resolves to 107.162.162.40
107.162.162.40:443: OPEN
TLS is available: True
Connecting to port 443 using HTTPS
Reading HTTP headers for www.f5.com
Queueing TLS scans (this might take a little while...)
0
Traceback (most recent call last):
File "/usr/local/bin/cryptonice", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/cryptonice/main.py", line 104, in main
output_data, hostname = scanner_driver(input_data)
File "/usr/local/lib/python3.8/dist-packages/cryptonice/scanner.py", line 427, in scanner_driver
tls_data = tls_scan(ip_address, str_host, commands_to_run, port)
File "/usr/local/lib/python3.8/dist-packages/cryptonice/modules/gettls.py", line 273, in tls_scan
preferred_cipher_suite = ssl2_result.cipher_suite_preferred_by_server
AttributeError: 'CipherSuitesScanResult' object has no attribute 'cipher_suite_preferred_by_server'

Tried this and getting this error, what is missing here...

Windows release 1.0.6 is missing root certs

The current installer does not package up the required PEM files in order to perform certificate checks.

Begin populating ReadTheDocs documentation

All errors sent to JSON should be written in nested JSON block

Update help to include volume/mount instructions

Can someone please update the docker related help to include instructions for how to mount or use volumes to get to the json output?

Multiple targets always output results to first target name

If scanning muliple targets such that

"targets": ["www.f5.com", "f5.com", "www.f5labs.com", "f5labs.com"]

All results will be written to the name of the first target, i.e. www.f5.com.json

Require ability to output JSON to specific directory

All JSON output is currently written to the directory from which Cryptonice is being run.

Dependency on pycurl requires libcurl devel packages outside of pip

Hello,

I went to install this on a CentOS 7.6 server today for testing and found that it has a dependency outside of pip via pycurl, which I believe is libcurl-devel. I'm waiting for my admin to install the package for me to verify, but it could be worthwhile to mention in any installation guides.

$ pip install cryptonice
Collecting cryptonice
  Using cached https://files.pythonhosted.org/packages/ab/5d/a333344eadfa3d357ce94553832ba343a5396ade129c14e0d51665c7c514/cryptonice-0.1.19-py3-none-any.whl
Collecting sslyze>=2.1.4 (from cryptonice)
  Using cached https://files.pythonhosted.org/packages/00/fb/56ccb9c9ab28027f9cedc5b80f3a66a42b73e1310c42eae15da01418c1de/sslyze-2.1.4.tar.gz
Collecting dnspython>=1.16.0 (from cryptonice)
  Using cached https://files.pythonhosted.org/packages/90/49/cb426577c28ca3e35332815b795a99e467523843fc83cc85ca0d6be2515a/dnspython-2.0.0-py3-none-any.whl
Collecting http-client>=0.1.21 (from cryptonice)
  Using cached https://files.pythonhosted.org/packages/94/e5/2e8134353629a91854c50a228f1624e21985aca1abc9d0ef0ce25bf5f389/http_client-0.1.22.tar.gz
Collecting urllib3>=1.25.9 (from cryptonice)
  Using cached https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl
Collecting ipaddress>=1.0.22pathlib~=1.0.1 (from cryptonice)
  Using cached https://files.pythonhosted.org/packages/c2/f8/49697181b1651d8347d24c095ce46c7346c37335ddc7d255833e7cde674d/ipaddress-1.0.23-py2.py3-none-any.whl
Collecting nassl<2.3.0,>=2.2.0 (from sslyze>=2.1.4->cryptonice)
  Downloading https://files.pythonhosted.org/packages/1c/59/2aa9ab432a09e0814231b52cd2dd030d7b308d7288ffcc8493a2bdb957db/nassl-2.2.0-cp36-cp36m-manylinux1_x86_64.whl (3.1MB)
    100% |████████████████████████████████| 3.1MB 3.3MB/s
Collecting cryptography==2.5 (from sslyze>=2.1.4->cryptonice)
  Using cached https://files.pythonhosted.org/packages/98/71/e632e222f34632e0527dd41799f7847305e701f38f512d81bdf96009bca4/cryptography-2.5-cp34-abi3-manylinux1_x86_64.whl
Collecting tls-parser<1.3.0,>=1.2.0 (from sslyze>=2.1.4->cryptonice)
  Downloading https://files.pythonhosted.org/packages/66/4e/da7f727a76bd9abee46f4035dbd7a4711cde408f286dae00c7a1f9dd9cbb/tls_parser-1.2.2.tar.gz
Collecting dataclasses (from sslyze>=2.1.4->cryptonice)
  Downloading https://files.pythonhosted.org/packages/e1/d2/6f02df2616fd4016075f60157c7a0452b38d8f7938ae94343911e0fb0b09/dataclasses-0.7-py3-none-any.whl
Collecting pycurl (from http-client>=0.1.21->cryptonice)
  Using cached https://files.pythonhosted.org/packages/ef/05/4b773f74f830a90a326b06f9b24e65506302ab049e825a3c0b60b1a6e26a/pycurl-7.43.0.5.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/tmp/pip-install-14reoqxd/pycurl/setup.py", line 234, in configure_unix
        stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      File "/usr/lib64/python3.6/subprocess.py", line 709, in __init__
        restore_signals, start_new_session)
      File "/usr/lib64/python3.6/subprocess.py", line 1344, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'curl-config': 'curl-config'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-14reoqxd/pycurl/setup.py", line 961, in <module>
        ext = get_extension(sys.argv, split_extension_source=split_extension_source)
      File "/tmp/pip-install-14reoqxd/pycurl/setup.py", line 623, in get_extension
        ext_config = ExtensionConfiguration(argv)
      File "/tmp/pip-install-14reoqxd/pycurl/setup.py", line 101, in __init__
        self.configure()
      File "/tmp/pip-install-14reoqxd/pycurl/setup.py", line 238, in configure_unix
        raise ConfigurationError(msg)
    __main__.ConfigurationError: Could not run curl-config: [Errno 2] No such file or directory: 'curl-config': 'curl-config'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-14reoqxd/pycurl/
You are using pip version 10.0.1, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

JSON serialization error

Some scans which result in intermittent errors produce output in the JSON which is unserializable. e.g.

'commands_with_errors': {'tls_1_3_early_data': <traceback.TracebackException object at 0x7ffa80788f50>}},

Certain implies values from apps.json aren't valid

Hello,

I ran the tool today against a server using IIS and got the following error.

(cryptonice) My Computer: cryptonice me$ cryptonice MyGreatIISSite.tld
Pre-scan checks
-------------------------------------
Scanning MyGreatIISSite.tld on port 443...
Analyzing DNS data for MyGreatIISSite.tld
Fetching additional records for MyGreatIISSite.tld
MyGreatIISSite.tld resolves to <IP>
<IP>:443: OPEN
TLS is available: True
Connecting to port 443 using HTTPS
Reading HTTP headers for MyGreatIISSite.tld
Traceback (most recent call last):
  File "/cryptonice/bin/cryptonice", line 10, in <module>
    sys.exit(main())
  File "/cryptonice/lib/python3.8/site-packages/cryptonice/__main__.py", line 53, in main
    output_data, hostname = scanner_driver(input_data)
  File "/cryptonice/lib/python3.8/site-packages/cryptonice/scanner.py", line 378, in scanner_driver
    redirection_results, http_data = get_http(ip_address, host_sni, port, target_tlsopen, http_body,
  File "/cryptonice/lib/python3.8/site-packages/cryptonice/modules/gethttp.py", line 402, in get_http
    elements = wappalyze(webpage)
  File "/cryptonice/lib/python3.8/site-packages/cryptonice/modules/gethttp.py", line 134, in wappalyze
    cats = apps[app_name]['cats']
KeyError: 'IIS\\;confidence:50'

I found that the implies field sometimes includes a confidence value, but that doesn't actually match the main key in the apps.json file. I worked around it by adding a few lines to strip off the confidence value and get a app_name variable that matches something in the json file.

Issue with Docker on Raspberry PI 4

docker run f5labs/cryptonice www.f5.com
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
standard_init_linux.go:219: exec user process caused: exec format error

ServerNetworkLocationViaDirectConnection Error

Getting this error after installation. Double checked the dependencies and cannot see anything missing.
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/cryptonice", line 5, in
from cryptonice.main import main
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptonice/main.py", line 1, in
from cryptonice.scanner import scanner_driver
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptonice/scanner.py", line 13, in
from .gettls import tls_scan
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptonice/gettls.py", line 4, in
from sslyze import (
ImportError: cannot import name 'ServerNetworkLocationViaDirectConnection' from 'sslyze' (/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sslyze/init.py)

Add recommendations functionality

cryptonice should produce recommendations based on vulnerabilities, weaknesses and missing features (e.g. DNS CAA not configured). An easy option would be to simply output during the console output (in scanner.py), however we may also want these recommendations in the JSON file.

Therefore, we should create a new "recommendations" dictionary option in each scanning module. All "recommendations" sections can then be read in and It Not None then output to the console.

Display the certificate issuer name to the user in the report.

Adding the ability to display the certificate issuer name appears trivial, and the data winds up in the JSON as well.

#14

CAA records are not properly checked for records at the domain level

We have our CAA records set up for vectorvms.com to allow only Comodo to issue SSL certificates for our domain. When using cryptonice we get the following:

RECOMMENDATIONS

Low - CAA Consider creating DNS CAA records to prevent accidental or malicious certificate issuance.

even though Qualys scans show no issues with our CAA record.
https://www.ssllabs.com/ssltest/analyze.html?d=vms.vectorvms.com

f5-labs / cryptonice Goto Github PK

cryptonice's People

Contributors

Stargazers

Watchers

Forkers

cryptonice's Issues

System information

$cryptonice f5.com Pre-scan checks

RECOMMENDATIONS

Recommend Projects

Recommend Topics

Recommend Org

Jobs

$cryptonice f5.com
Pre-scan checks