westpointltd / tls_prober Goto Github PK

I've been thinking whether we should add probes with SSLv2 Hello protocol probes, that is SSLv2 hello that advertises support for TLSv1.0 or TLSv1.2.

also SSLv2 parsing code will likely have completely different bugs from the SSLv3/TLS parsing code, so in theory it should provide good signal to differentiate implementations from each-other.

It's also code that is hard to disable, from what I know, only Java provides explicit trigger for that, and OpenSSL can do it, but only if the application enables support just for one specific version of TLS not the universal protocol handler.

Fingerprints for Mozilla Network Security Services, often used by Red Hat and Oracle as server, are missing.

(I plan to add them later, once I finish work on new probes, adding it here as a reminder)

Erlang OTP includes TLS implementation, check its behaviour

Would be great to have tls_prober be installable from PyPI, via pip, in virtualenv with ease, etc.

https://packaging.python.org/tutorials/packaging-projects/

create a probe that sends a warning level alert before sending the Client Hello

There are no fingerprints for LibreSSL, it's misdetected (with confidence in ~80% range) as OpenSSL.

Currently servers that only support DH/ECDH will not fingerprint properly.

according to F5 documentation, ClientHello messages fragmented to less than 8 bytes are not recognised as TLS packets.

Unfortunately the SplitHelloRecords tests and SplitHelloPackets use a 10/n-10 split which won't detect that.

Create a test with 1/n-1 split for finding it.

see also: drwetter/testssl.sh#1113

cryptlib is a commonly used TLS library used in embedded applications, so it would be especially useful to have its fingerprints.

Unfortunately, the documentation and code examples are... unavailable or hard to find. So setting up a "known good" server is non-trivial.

It would be nice to have some option like with the unix head command to limit the display of guesses t the top N entries.

E.g. have:
./probe.py -5 example.org
return the top 5 most likely implementations for example.org

If missing or 0 return full list as usual:
./probe.py example.org
./probe.py -0 example.org
Both return the full list of guesses in decending order of likeliness.

Botan fingerprints are missing.

TLS 1.3 adds multiple extensions and new ciphers

• the generic probe should include the extensions and ciphers necessary for TLS 1.3 negotiation
  • note that support for different draft versions can be used to differentiate between versions of the same server
• handling malformed TLS 1.3 extensions can be different between implementations

given that the negotiated version in the newest draft is sent in extension, so the format will need to be extended to provide that value to the client

in older draft versions HelloRetryRequest is a separate handshake message, I wonder if we should extend the parser to be able to handle it...

I'm not entirely sure what's going on with the fingerprinting of my blog, but my purse told me I did not by any FortiOS devices for https://blog.benny-baumann.de - instead it's running a nginx on Debian (testing).

$ ./prober.py blog.benny-baumann.de
FortiOS v5.2.2,build642 (GA)    12
openssl-1.0.1h default source build 7
openssl-1.0.1c default source build 7
openssl-1.0.1e default source build 7
F5 FirePass 6.1.0 URM-6.10-20091009 7

$ ./prober.py -l blog.benny-baumann.de
    ZeroHelloVersion    error:ECONNRESET|
      BadContentType    *(5454)record:type(48)|error:Unexpected EOF receiving record header - server closed connection|
        SNIEmptyName    *(301)alert:DecodeError:fatal|
   SplitHelloRecords    *(5454)record:type(48)|error:Unexpected EOF receiving record header - server closed connection|
         EmptyRecord    error:ECONNRESET|
RecordLengthUnderflow   writeerror:ECONNRESET|
          Heartbleed    *(301)alert:HandshakeFailure:fatal|
 BadHandshakeMessage    *(301)alert:HandshakeFailure:fatal|
     NormalHandshake    *(301)alert:HandshakeFailure:fatal|
  OnlyECCipherSuites    *(301)handshake:ServerHello(301)|*(301)handshake:Certificate|*(301)handshake:ServerKeyExchange|*(301)handshake:ServerHelloDone|
           NoCiphers    *(301)alert:IllegalParameter:fatal|
  VeryHighTLSVersion    error:ECONNRESET|
VeryHighHelloVersion    *(303)handshake:ServerHello(303)|*(303)handshake:Certificate|*(303)handshake:ServerHelloDone|
   DoubleClientHello    *(301)alert:HandshakeFailure:fatal|
           Heartbeat    *(301)alert:HandshakeFailure:fatal|
      HighTLSVersion    error:ECONNRESET|
    HighHelloVersion    *(303)handshake:ServerHello(303)|*(303)handshake:Certificate|*(303)handshake:ServerHelloDone|
   SplitHelloPackets    *(301)alert:HandshakeFailure:fatal|
EmptyChangeCipherSpec   *(301)alert:HandshakeFailure:fatal|
RecordLengthOverflow    error:timeout
    ChangeCipherSpec    *(301)alert:HandshakeFailure:fatal|
        SNIWrongName    *(301)alert:HandshakeFailure:fatal|
         SNILongName    *(301)alert:UnrecognizedName:fatal|
      ZeroTLSVersion    error:ECONNRESET|

The server is running:

# openssl version
OpenSSL 1.0.1k 8 Jan 2015

# dpkg -s nginx-full | grep -ix '[PV].*'
Package: nginx-full
Version: 1.6.2-5

The basic template for the nginx SSL config basically looks like:

ssl_protocols               TLSv1.2 TLSv1.1 TLSv1;
ssl_prefer_server_ciphers   on;
ssl_ciphers                 HIGH:+CAMELLIA256:!eNull:!aNULL:!ADH:!MD5:-RSA+AES+SHA1:!RC4:!DES:!3DES:!SEED:!EXP:!AES128:!CAMELLIA128;

ssl_session_cache           shared:SSL:16m;
ssl_session_timeout         5m;
ssl_session_tickets         on;
ssl_session_ticket_key      /etc/nginx/sessions.d/current.key;

ssl_client_certificate      /etc/ssl/certs/ca-certificates.pem;
ssl_trusted_certificate     /etc/nginx/trusted-ca.pem;

#ssl_verify_client           optional_no_ca;
ssl_verify_client           off;
ssl_verify_depth            5;

ssl_stapling                on;
ssl_stapling_verify         on;

ssl_dhparam                 /etc/ssl/dh/server13337.dh.pem;

#ssl_certificate             /etc/ssl/public/   $server_name.crt;
#ssl_certificate_key         /etc/ssl/private/  $server_name.key;

The used certificates are 8192 bit RSA. The used DH parameters are as indicated in the filenames.

there is specific behaviour in old versions of NSS in regards to signature algorithms handling:
https://bugzilla.mozilla.org/show_bug.cgi?id=1317857#c18

Currently there's just one fingerprint file for Go TLS implementation and it's without version.

Collect fingerprints from multiple versions, remove the existing one.

Current probes:

• 0 — server_name [RFC6066] #24 #23
• 1 — max_fragment_length [RFC6066] #24
• 2 — client_certificate_url [RFC6066] #24
• 3 — trusted_ca_keys [RFC6066] #24
• 4 — truncated_hmac [RFC6066] #24
• 5 —status_request [RFC6066] #24
• 6 — user_mapping [RFC4681] #34
• 7 — client_authz [RFC5878] #34
• 8 — server_authz [RFC5878] #34
• 9 — cert_type [RFC6091] #34
• 10 — supported_groups (formerly "elliptic_curves") [RFC4492][RFC7919] #34
• 11 — ec_point_formats [RFC4492] #34
• 12 — srp [RFC5054] #34
• 13 — signature_algorithms [RFC5246] #34
• 14 — use_srtp [RFC5764] #34
• 15 — heartbeat [RFC6520] #34
• 16 — application_layer_protocol_negotiation [RFC7301] #34
• 17 — status_request_v2 [RFC6961] #34
• 18 — signed_certificate_timestamp [RFC6962] #34
• 19 — client_certificate_type [RFC7250] #34
• 20 — server_certificate_type [RFC7250] #34
• 21 — padding [RFC7685] #34
• 22 — encrypt_then_mac [RFC7366] #34
• 23 — extended_master_secret [RFC7627] #34
• 24 — token_binding [draft-ietf-tokbind-negotiation]
• 25 — cached_info [RFC7924] #34
• 35 — SessionTicket TLS [RFC4507] #34
• 13172 — supports_npn [draft-agl-tls-nextprotoneg-04] #34
• 62208 — tack [draft-perrin-tls-tack-02] #34
• 65281 — renegotiation_info [RFC5746] #22

Even if a library allows to change which extensions are supported (many don't), and those settings are exposed by the applications that use the TLS libraries (again, many don't do that), most users still focus only on supported ciphers, protocol versions and maybe (EC|FF)DH parameters.

So I think that detecting which extensions are supported by server would be a very strong signal for probing servers.

As to how to detect if a particular extension is supported? Send an invalid formatting of the extension and see if server aborts connection.

Is this something you'd like to see @richmoore?

Hello,

I've been encountering socket timeout exception.

[How to Produce Error]

$ ./prober.py www.google.com
Traceback (most recent call last):
  File "./prober.py", line 420, in <module>
    main()
  File "./prober.py", line 389, in main
    results = probe(args[0], opts.port, opts.starttls, opts.probe)
  File "./prober.py", line 329, in probe
    result = probe.probe(ipaddress, port, starttls)
  File "/home/onur/tp/tls_prober/probes.py", line 124, in probe
    sock = self.connect(ipaddress, port, starttls)
  File "/home/onur/tp/tls_prober/probes.py", line 49, in connect
    s.connect((ipaddress, port))
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.timeout: timed out

[Posibble Fix]
I believe problem occurs in here. Looks like that line is put outside of try block by accident.

I would like to send a pull request for this patch if it is okay.

Thank you.