tintinweb / scapy-ssl_tls Goto Github PK
View Code? Open in Web Editor NEWSSL/TLS layers for scapy the interactive packet manipulation tool
License: GNU General Public License v2.0
SSL/TLS layers for scapy the interactive packet manipulation tool
License: GNU General Public License v2.0
I see you have a POC of TSL decryption which seems quite nice.
I just found this project but it seems really interesting, would it be possible to build a generic SSL/TLS MITM Proxy with this?
What would be a good way of approaching this?
integration suite is missing:
So sorry for opening too many issues , but I have one more.
I am unable to access parsed SSL/TLS layers from scapy-ssl_tls.
To duplicate here is what I have tested with the security_scanner.py example:
on one terminal I ran this:
while true;do timeout 4 openssl s_client -connect 151.101.49.140:443;done
On a different tab:
There are no matches,this is all I get:
scapy-ssl_tls # python2.7 examples/security_scanner.py sniff 151.101.49.140 443
WARNING: No route found for IPv6 destination :: (no default route?)
An example implementation of a passive TLS security scanner with custom starttls support:
TLSScanner() generates TLS probe traffic (optional)
TLSInfo() passively evaluates the traffic and generates events/warning
[*] [passive] Scanning in 'sniff' mode for ('151.101.49.140', 443) on eth0...
To duplicate the problem, I've tried emulating the _process_packet callback with no bpf filter set to see if any SSL/TLS layers ever match:
import sys, os,traceback
from pprint import pprint
import concurrent.futures
try:
from scapy.all import get_if_list, sniff, IP, TCP
except ImportError:
from scapy import get_if_list, sniff, IP, TCP
try:
# This import works from the project directory
basedir = os.path.abspath(os.path.join(os.path.dirname(__file__),"../"))
sys.path.append(basedir)
from scapy_ssl_tls.ssl_tls import *
from scapy_ssl_tls.ssl_tls_crypto import x509_extract_pubkey_from_der
except ImportError:
# If you installed this package via pip, you just need to execute this
from scapy.layers.ssl_tls import *
from scapy.layers.ssl_tls import x509_extract_pubkey_from_der
import socket
from collections import namedtuple
import time
def _process_packet(pkt):
if pkt is None:
return
if not pkt.haslayer('SSL') and not (pkt.haslayer('TLSRecord') or pkt.haslayer('SSLv2Record')):
return
if pkt.haslayer('SSL'):
records= pkt['SSL'].records
print ("SSL!")
else:
records = [pkt]
for r in records:
if r.haslayer('TLSClientHello'):
print('TLSClientHello')
elif r.haslayer('TLSServerHello'):
print('TLSServerHello')
elif r.haslayer('TLSRecord'):
print('TLSRecord')
elif r.haslayer('SSLv2Record'):
print('SSLv2Record')
elif r.haslayer('TLSAlert'):
print('TLSAlert')
elif r.haslayer('SSLv2ClientHello'):
print('SSLv2ClientHello')
elif r.haslayer('TLSExtHeartbeat'):
print('TLSExtHeartbeat')
elif r.haslayer('TLSCertificateList'):
print('TLSCertificateList')
elif r.haslayer('TLSHandshake'):
print('TLSHandshake')
elif r.haslayer('TLSFinished'):
print('TLSFinished')
def main():
try:
sniff(prn=_process_packet,filter="",store=0)
except Exception as e:
print(traceback.format_exc())
if __name__=="__main__":
main()
I have also tried unquoting the haslayer() and pkt[] arguments:
import sys, os,traceback
from pprint import pprint
import concurrent.futures
try:
from scapy.all import get_if_list, sniff, IP, TCP
except ImportError:
from scapy import get_if_list, sniff, IP, TCP
try:
# This import works from the project directory
basedir = os.path.abspath(os.path.join(os.path.dirname(__file__),"../"))
sys.path.append(basedir)
from scapy_ssl_tls.ssl_tls import *
from scapy_ssl_tls.ssl_tls_crypto import x509_extract_pubkey_from_der
except ImportError:
# If you installed this package via pip, you just need to execute this
from scapy.layers.ssl_tls import *
from scapy.layers.ssl_tls import x509_extract_pubkey_from_der
import socket
from collections import namedtuple
import time
def _process_packet(pkt):
if pkt is None:
return
if not pkt.haslayer(SSL) and not (pkt.haslayer(TLSRecord) or pkt.haslayer(SSLv2Record)):
return
if pkt.haslayer(SSL):
records= pkt[SSL].records
else:
records = [pkt]
for r in records:
if r.haslayer(TLSClientHello):
print('TLSClientHello')
elif r.haslayer(TLSServerHello):
print('TLSServerHello')
elif r.haslayer(TLSRecord):
print('TLSRecord')
elif r.haslayer(SSLv2Record):
print('SSLv2Record')
elif r.haslayer(TLSAlert):
print('TLSAlert')
elif r.haslayer(SSLv2ClientHello):
print('SSLv2ClientHello')
elif r.haslayer(TLSExtHeartbeat):
print('TLSExtHeartbeat')
elif r.haslayer(TLSCertificateList):
print('TLSCertificateList')
elif r.haslayer(TLSHandshake):
print('TLSHandshake')
elif r.haslayer(TLSFinished):
print('TLSFinished')
def main():
try:
sniff(prn=_process_packet,filter="",store=0)
except Exception as e:
print(traceback.format_exc())
if __name__=="__main__":
main()
I still don't get any output, I think the scapy-ssl_tls layer isn't adding the fields right when dissecting the packet? (I'm still learning how this works)
I will keep reading your source code to figure out where this could be going wrong so I can help as well.
Best regards.
Have you considered porting this to python3 and scapy-python3 (http://github.com/phaethon/scapy)?
I am ready to contribute 50% of the effort.
I use version 1.2.3 of scapy and when i tried to use that extension it gives me
Attribute error: algorithms
The fix is changing the
XFieldLenField("length", None, length_of="algorithms", fmt="H"),
to
XFieldLenField("length", None, length_of="algs", fmt="H"),
in ssl_tls.py : row420
Hi there tintinweb,
I have some questions regarding scapy-ssl_tls I'd rather make privately (if you're willing to answer, of course). Could you post some contact information please? (email, IRC, ..)
ssl_tls.py is the entrypoint for all ssl_tls related layers and functionality and should therefore allow to access the TLSSessionCTX/.... without having to explicitly import it from ssl_tls_crypto. We should also make sure that one can still use the layers without having to install pyCrypto (try_catch:ImportError).
First let me say terrific work with this!
The problem i ran into was while running the example client script using ECDHE . It works fine with certain servers and does not with others. Ostensibly this problem appears to be predicated on how the server responds. From packet captures( attached , please open with wireshark) when server is replying with server hello , server key exchange in separate record layers , the ssl handshake completes successfully and application data is exchanged. However when a server replies with server hello, certificate , server key exchange all encapsulated in one record layer as evidenced in the first tcp stream in the capture file, the client script errors out. At this time , from viewing the tls_ctx, i see all crypto.server.ecdh attributes are none and from the code it looks like if p.haslayer(tls.TLSServerECDHParams) condition is not True.
params.negotiated.version=TLS_1_2
params.negotiated.ciphersuite=ECDHE_RSA_WITH_AES_256_CBC_SHA
params.negotiated.key_exchange=ECDHE
params.negotiated.encryption=('AES', 32, 'CBC')
params.negotiated.mac=SHA
params.negotiated.compression=NULL
crypto.client.enc=None
crypto.client.dec=None
crypto.server.enc=None
crypto.server.dec=None
crypto.client.rsa.privkey=None
crypto.client.rsa.pubkey=None
crypto.server.rsa.privkey=None
crypto.server.rsa.pubkey=<_RSAobj @0x7f8d5b7f7b00 n(2048),e>
crypto.client.dsa.privkey=None
crypto.client.dsa.pubkey=None
crypto.server.dsa.privkey=None
crypto.server.dsa.pubkey=None
crypto.client.dh.x=None
crypto.client.dh.y_c=None
crypto.server.dh.p=None
crypto.server.dh.g=None
crypto.server.dh.x=None
crypto.server.dh.y_s=None
crypto.client.ecdh.curve_name=None
crypto.client.ecdh.priv=None
crypto.client.ecdh.pub=None
crypto.server.ecdh.curve_name=None
crypto.server.ecdh.priv=None
crypto.server.ecdh.pub=None
crypto.session.encrypted_premaster_secret=None
crypto.session.premaster_secret="\x03\x03\xda\x8e`\xb1.\xf9)\x1c#x\xef\x1d\x0eE\xfb\xe1\xe2\xa2\xd33y\xc1\xd3';\xb2\x9c,S\x04\xbf%\x88\x89\xd9\xe3\xc8\xb2r\xe0\xc4\x18\xc3\xd0z\x10"
crypto.session.master_secret=None
crypto.session.randombytes.client='X6j\xfc\x15n\xd6/#\xf4\x0e%\xd3\xfb\x07\xde;\xc2\xd0\xd0d\x99\x84\xd4\xe0\xc2\x18J\xff\x97Sm'
crypto.session.randombytes.server="X6j\xfcK\x1d?!\x1fp\xceg}\x1dA\xa3\xf2\x89\xed4\xd1\xa5k{>\xf0\xfd\xd4'\x9f\xfe\xf6"
crypto.session.key.client.mac=None
crypto.session.key.client.encryption=None
crypto.session.key.cllient.iv=None
crypto.session.key.server.mac=None
crypto.session.key.server.encryption=None
crypto.session.key.server.iv=None
crypto.session.key.length.mac=None
crypto.session.key.length.encryption=None
crypto.session.key.length.iv=None
Traceback (most recent call last):
File "ecdhessl_scapytest.py", line 62, in
tls_client(server)
File "ecdhessl_scapytest.py", line 49, in tls_client
tls_client_key_exchange(sock)
File "ecdhessl_scapytest.py", line 30, in tls_client_key_exchange
client_key_exchange = TLSRecord(version=tls_version) / TLSHandshake() / sock.tls_ctx.get_client_kex_data()
File "/usr/local/lib/python2.7/dist-packages/scapy_ssl_tls/ssl_tls_crypto.py", line 585, in get_client_kex_data
return tls.TLSClientKeyExchange(ctx=self) / tls.TLSClientECDHParams(data=self.get_client_ecdh_pubkey(val))
File "/usr/local/lib/python2.7/dist-packages/scapy_ssl_tls/ssl_tls_crypto.py", line 566, in get_client_ecdh_pubkey
ec_curve = ec_reg.get_curve(self.crypto.server.ecdh.curve_name)
File "/usr/local/lib/python2.7/dist-packages/tinyec/registry.py", line 95, in get_curve
if name.lower() == k.lower():
AttributeError: 'NoneType' object has no attribute 'lower'
Thanks!
583.pcap.zip
TLS protocol is properly parsed for package no. 3 of file RSA_WITH_AES_128_CBC_SHA.pcap but not for my capture (firefox talking TLS 1.2 to proxy).
For instance, code rdpcap('583.pcap')[0].show()
shows TLS as raw bytes.
However, I managed to forcefully parse the package:
p=rdpcap('583.pcap')[0]
s=SSL(str(p))
r=str(p[TCP].payload)
s.do_dissect(r)
s.show()
I get output that looks good:
###[ SSL/TLS ]###
\records \
|###[ TLS Record ]###
| content_type= handshake
| version = TLS_1_0
| length = 0x200
|###[ TLS Handshake ]###
| type = client_hello
| length = 0x1fc
|###[ TLS Client Hello ]###
| version = TLS_1_2
| gmt_unix_time= 4161341682
| random_bytes= ...
Probably TLS autodetection does not work while do_dissect()
method works fine.
I use commit 628ff4e installed with method 2 over gentoo'ish scapy-2.3.2 if it matters.
Hi,
when running
sudo python server_rsa.py
and connecting to it, I get the following error:
Traceback (most recent call last):
File "server_rsa.py", line 32, in <module>
version = r[TLSClientHello].version
File "/usr/lib64/python2.7/site-packages/scapy/packet.py", line 817, in __getitem__
raise IndexError("Layer [%s] not found" % lname)
IndexError: Layer [TLSClientHello] not found
I'm using the following versions:
$ pip list | grep scapy
scapy (2.3.2)
scapy-ssl-tls (1.2.3.2)
Is there any quick fix for this problem?
Kind regards, Jan
I installed scapy-ssl_tls as stated in the README and in general it seems to work well.
But when I write
from scapy.all import *
I get
Import Error - most likely due to missing pycrypto libraries - disabling crypto functionality
ImportError('cannot import name TLSHandshake',)
It says it's probably due to missing pycrypto library but I already installed this dependency.
# pip freeze
NetfilterQueue==0.3
pycrypto==2.6.1
scapy==2.3.1
Is something else missing? Maybe it would be a good idea to include a requirements.txt or at least name all dependencies in the README.
Apart of that: Thank you very much for this extension! It's really missing in scapy.
First let me say that this is an awesome project, I have a lot of things I can't wait to use it for.
However, as I play around, I've run into an issue with 'sessionctx_sniffer' example. I ran a wireshark capture on a connection from the 'full_rsa_connection_with_application_data' example to an openssl server I was running on localhost. I then passed the pcap and private key to the sniffer example. However there is no output after the "load servers private key".
After some investigation, this appears to be because while the packets appear to have an SSL/TLS layer when I show
them, haslayer(SSL)
returns False
. This causes this line to be an empty list. My suspicion is that the reassembly is damaging the packets in some way, but I have not been able to find the root of the problem.
I can provide the capture and private key (I just generated it for this, it doesn't need to be kept secret) if that would be helpful.
EDIT: This is some kind of import error. If I install through setup.py
and remove the imports from the project dir, this works.
Hi,
I tried executing the full_rsa_connection_with_application_data.py file given in the examples directory, with a correct server and port as command line arguments, but ended up receiving a Name Error, stating "global name 'TLSSocket' is not defined".
I am not clear as to what am I missing here? Any ideas?
Besides, I am able to run the other files-client_hello_valid.py without any issues.
Thanks,
Hi,
I am getting the following error: Traceback (most recent call last): File "client_hello_with_session_ticket.py", line 38, in <module> tls_ctx.get_client_kex_data()]) File "/media/sf_TLS_Tutorial/scapy-ssl_tls/scapy_ssl_tls/ssl_tls_crypto.py", line 577, in get_client_kex_data return tls.TLSClientKeyExchange() / tls.TLSClientECDHParams(data=self.get_client_ecdh_pubkey(val)) TypeError: unsupported operand type(s) for /: 'TLSClientKeyExchange' and 'TLSClientECDHParams' '
when running the following code: `
if len(sys.argv) > 2:
server = (sys.argv[1], int(sys.argv[2]))
else:
server = ("127.0.0.1", 8443)
host = (sys.argv[1], int(sys.argv[2]))
version = TLSVersion.TLS_1_2
cipher = TLSCipherSuite.ECDHE_RSA_WITH_AES_256_CBC_SHA
ticket = ""
master_secret = ""
with TLSSocket(socket.socket(), client=True) as tls_socket:
tls_socket.connect(host)
tls_ctx = tls_socket.tls_ctx
pkt = TLSRecord() / \
TLSHandshakes(handshakes=[TLSHandshake() /
TLSClientHello(version=version,
cipher_suites=[cipher],
extensions=[TLSExtension() /
TLSExtSessionTicketTLS(data="")])])
# pkt.show()
tls_socket.sendall(pkt)
resp = tls_socket.recvall()
# resp.show()
client_key_exchange = TLSRecord(version=version) / \
TLSHandshakes(handshakes=[TLSHandshake() /
tls_ctx.get_client_kex_data()])
# client_key_exchange.show()
client_ccs = TLSRecord(version=version) / TLSChangeCipherSpec()
tls_socket.sendall(TLS.from_records([client_key_exchange, client_ccs]))
tls_socket.sendall(TLSHandshakes(handshakes=[TLSHandshake() /
TLSFinished(data=tls_socket.tls_ctx.get_verify_data())]))
server_finished = tls_socket.recvall()
# server_finished.show()
ticket = server_finished[TLSSessionTicket].ticket
tls_socket.sendall(TLSPlaintext(data="GET / HTTP/1.1\r\nHOST: localhost\r\n\r\n"))
resp = tls_socket.recvall()
# resp.show()
master_secret = tls_ctx.master_secret`
It is a very slight variation of your client_hello_with_ticket.py example.
I am running python2.7, if that makes a difference.
Thank you
Hi there,
I am having trouble getting the decrypted "application data" from a pcap file using "scapy-ssl_tls/examples/sessionctx_sniffer.py"
I get "<TLSCiphertext data=...(the encrypted data)" BUT I am looking for "decrypted record"
All my parameters for the main work, i.e. my pcap file, .key file, and host and port numbers.
Please respond at your earliest convenience.
Thank you
pip install is pulling scapy 2.3.3, which doesn't work, leading to the same behaviour as #69
$ pip install --user scapy-ssl_tls
Collecting scapy-ssl_tls
Downloading scapy-ssl_tls-1.2.3.tar.gz (117kB)
100% |โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ| 118kB 1.0MB/s
Collecting scapy (from scapy-ssl_tls)
Downloading scapy-2.3.3.tgz (1.4MB)
100% |โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ| 1.4MB 234kB/s
Requirement already satisfied (use --upgrade to upgrade): pycrypto in /usr/lib64/python2.7/site-packages (from scapy-ssl_tls)
Collecting tinyec (from scapy-ssl_tls)
Downloading tinyec-0.3.1.tar.gz
Installing collected packages: scapy, tinyec, scapy-ssl-tls
Running setup.py install for scapy
Running setup.py install for tinyec
Running setup.py install for scapy-ssl-tls
Successfully installed scapy scapy-ssl-tls tinyec
Hi,
Could you please tell me how did you exchange the keys in this line
client_key_exchange = TLSRecord(version=tls_version) / TLSHandshake() / sock.tls_ctx.get_client_kex_data()
I couldn't understand what does (tls_ctx) mean!
I have my keys and I have the site's keys since it's on my local machine. I tried many times to encrypt a pre-master with the site's public key and sent it over, but it kept failing every time. I tried to deal with RSA and Diffie-Hellman mechanisms but none worked with me.
Hi,
Some poor (or old) SSL implementations are vulnerable to DOS by sending large overflow values in the Client Hello packet. Since these implementations don't support tcp window scaling and wait for the entire client hello packet (even though it might be much larger than 32kb) it can keep a connection busy even without completing a ssl handshake.
I tried doing this test using scapy ssl tls. I modified gmt_unix_time header to support variable length strings and did a client hello with a gmt_unix_time="a" * 10000000, but I am getting a Fragmentation Error as the packet exceeds 65535 bytes.
I would like to know if it is possible to send such large overflowing values or is it a limitation?
Thanks!
pycryptodome provides a drop in replacement for pycrypto.
It also supports GCM and CCM modes with the same interface, which would make it trivial to implement for us
I need to create a certificate request record. However, I tried various inputs for TLSCertificateRequest and got the error as below.
TLSRecord(version=version) /
TLSHandshakes(handshakes=[TLSHandshake() / TLSCertificateRequest(ca_dns=??????)])
Error:
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 1104, in getfield_and_val
raise AttributeError(attr)
AttributeError: dns
Hi again @alexmgr
I have a test case which requires me to send tlsrecord of > 2**19. I realize this is not rfc compliant which is sort of the objective of my test case.
I'm still parsing through the code to see what can be done to accommodate this - already made changes to MAX_LEN in tlsrecord and also tried changing length to unsigned long or unsigned int. When i make this format character chnage , example say i change code to XLenField("length", None, fmt="!L")] in fields_desc of tlsrecord class, things get messed up in general with all tlsrecord's. example a working benchmark client hello when code remains unchanged( i.e fmt="!H") hex dump looks like
0000 16 03 03 00 2d 01 00 00 29 03 03 58 73 d4 f0 4a
0010 b4 f2 a2 45 25 5c f5 c7 38 fc 13 b5 d1 f6 02 f2
0020 83 d2 2d 42 64 bc f5 60 9d f7 df 00 00 02 00 35
0030 01 00
and is of 116 bytes
when i make the change to unsigned long the client hello hex dump looks like
0000 16 03 03 00 00 00 2d 01 00 00 29 03 03 58 73 d5
0010 78 cf 28 5c fb 62 35 ff 98 ea 0a c4 49 a0 50 56
0020 bd 64 5f f9 18 65 31 ad 3f c5 12 66 98 00 00 02
0030 00 35 01 00
and is of 118 bytes
the problem with the non working one is the extra two bytes of 00 00 ( in bold) inserted. If not for that , the hexdump would be congruous to the working hexdump and should work fine.
I'm still perusing the code to understand it better and make correct adjustments , but in the mean time if you have any quick tips or suggestions , i would very much appreciate it
Thanks!
I have several cases in that I can't see the [ClientHello].extensions_length but I can see it
with wireshark. I've been looking for source code and this field is conditional and I think the condition doesn't work well.
I have attached several capture files with extension file chenged, from pcap to png
Best regards
Can I use your tools to view the encrypted data downloaded from a TLS server?
Hey hi,
The tool you folks have written is a great support for my work. I have been able to get a lot of work done because of this, but there's a tad bit more that I need it for.
During an SSL handshake, a master key is generated. From that master key 4 keys are generated.
Is there someway I could (a)either extract the master key and derive the 4 keys outside of the handshake, or (b)get the 4 keys once the connection has been done. I would prefer if (a) could be done.
Although I understand that this may not be done on grounds of security, but there's something I am working on where I need to regenerate these 4 keys.
Also, when we tried to somehow get the keys out, we got something like the following:
crypto.session.master_secret='Q\x96\x91\x11I\xce\xc52v\xa8%\x1a<i\xf7\xa3\x85>\xa4\xc0\xf8\xa8yx\xee\xc2@\xbfN>\x8e\xb5\xa6\xcf|R\xe5\xb8\xc7\x00A!\x1e\xba\xfd\x1ad\xac'
crypto.session.randombytes.client="U\xb0m\xdb\xe8J\xf3+\xbd\xe1\xe6\xa7\xad\xe9\x86\xfe\xbf#0's\xdc\xcb)\x88/2Z\xdd\x00k\xb9"
crypto.session.randombytes.server='U\xb0m\xd9\xc3S\x83I\x8f%\x95?\xf4\xf5\xddL\xa8l1\x8f\xdd\xd5\xe9\xddy\xc9z\x96n\xfe\xfd\xcc'
crypto.session.key.client.mac='\xc4\xcc\xd5\xd8)\xce\xbcO\xee\xea\xb6\xca~\x855w\xd2\x9c\xd9\xcc
crypto.session.key.client.encryption='\xad\xfe\x8aH\xceq\x0b\xce\r `\xf7K*_4'
crypto.session.key.cllient.iv='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
crypto.session.key.server.mac='-a\xc4\xdc\xb8\xb0\r\x0ck\xab_K\x93\xe4\x9b\xb4\x0b\x86\xeb\x1f'
crypto.session.key.server.encryption='\x99\xfb\xd5\x81I\xf5P\x1fE\x07(\xf6\x82\x9c\x91]'
crypto.session.key.server.iv='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
crypto.session.key.length.mac=20
crypto.session.key.length.encryption=16
crypto.session.key.length.iv=16
However, the characters therein don't look like hex [there exist characters like +, ~, which will not belong to the hex character set].
Any kind of help in this regard [(a) or (b), as stated above] would be appreciated.
Hi,
While scanning a site for sslv2 support I am seeing the following issue:
pkt = SSLv2Record()/SSLv2ClientHello(cipher_suites=cipherlist,
challenge='A'\*16)
t = socket.socket()
t.connect(target) #target=("clani.taborniki.si",443)
ts = TLSSocket(t,client=True)
ts.sendall(pkt)
resp = ts.recv(8192\*4)
resp = SSL(''.join(resp))
resp.show()
The TLS/SSL records in response are empty:
###[ SSL/TLS ]###
\records \
However, when I manually set the lengths of pkt like this:
pkt = SSLv2Record(length=(25+3\*len(cipherlist)))/SSLv2ClientHello(cipher_suites=cipherlist,
challenge='A'\*16,
challenge_length=16,
cipher_suites_length=3\*len(cipherlist)
)
The sslv2 connection is successful. The response (pasted partial) I get is:
###[ SSL/TLS ]###
\records \
|###[ SSLv2 Record ]###
| length = 0x580
| content_type= server_hello
|###[ SSLv2 Server Hello ]###
| session_id_hit= false
| certificate_type= x509
| version = SSL_2_0
| certificates_length= 0x55f
| cipher_suites_length= 0x6
| connection_id_length= 0x10
.........
Are the length fields in packet being set correctly ? Because when I do pkt.show()
, the length fields in all layers are None
. I thought this would be set by scapy before packet is transmitted but this does not appear to be the case ?
Also, this maybe happening with other protocols as well. This does not result in dropped connections always, but only in some site cases and not others.
Is their a way to set the lenghts in different layers dynamically ?
Thanks
adapt sessonctx_sniffer tls decryptor code due to recent changes in TLSSessionCtx
.
e.g.
- session.crypto.session.master_secret = None
+ session.master_secret = None
I am having trouble in doing a SSL renegotiation using the renegotiation_info extension of the client hello packet. While doing a SSL renegotiation I also want to be able to change some of the attributes of the client hello packet like cipher suites and elliptic curves. I am not very familiar with the scapy-ssl_tls tool. Can someone help me regarding this?
Hello
In the RFC(6066), the maximum length permitted for ServerNameIdication is 2^16-1, but when I try to assign a value with this length (capture[3][TLSServerName].data="a" * 65535) and then save this capture, I get this error:
Traceback (most recent call last):
File "change_packet.py", line 475, in
main()
File "change_packet.py", line 467, in main
capture_obj.extra_repair_capture()
File "change_packet.py", line 256, in extra_repair_capture
execfile("./extra_funcionality.py",variables)
File "./extra_funcionality.py", line 335, in
change_sni(capture[3],"b" * 65535)
File "./extra_funcionality.py", line 174, in change_sni
extensions_length += len(tls_extension)
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 297, in len
return len(self.str())
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 268, in str
return self.build()
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 330, in build
p = self.do_build()
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 319, in do_build
pkt = self.self_build()
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 310, in self_build
p = f.addfield(self, p, val)
File "/usr/local/lib/python2.7/dist-packages/scapy/fields.py", line 70, in addfield
return s+struct.pack(self.fmt, self.i2m(pkt,val))
File "/usr/local/lib/python2.7/dist-packages/scapy/fields.py", line 613, in i2m
x = len(pkt.payload)
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 297, in len
return len(self.str())
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 268, in str
return self.build()
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 330, in build
p = self.do_build()
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 319, in do_build
pkt = self.self_build()
File "/usr/local/lib/python2.7/dist-packages/scapy/packet.py", line 310, in self_build
p = f.addfield(self, p, val)
File "/usr/local/lib/python2.7/dist-packages/scapy/fields.py", line 70, in addfield
return s+struct.pack(self.fmt, self.i2m(pkt,val))
struct.error: 'H' format requires 0 <= number <= 65535
In the initial handshake, implementation seem to allow the use of stacked handshakes within a record. For example Java does it, and wireshark dissects it as such. We should handle that case. In this example, we fail to find the cert, since the stacked handshakes are not correctly parsed:
(scapy-ssl_tls)amoneger@home contrib/scapy-ssl_tls (tls-socket) ยป tests/integration/java_tls_server.sh
Connected to server: ('127.0.0.1', 8443)
###[ SSL/TLS ]###
\records \
|###[ TLS Record ]###
| content_type= handshake
| version = TLS_1_0
| length = 0x4ca
|###[ TLS Handshake ]###
| type = server_hello
| length = 0x46
|###[ TLS Server Hello ]###
| version = TLS_1_0
| gmt_unix_time= 1433481769
| random_bytes= 'x\x01T\xad\xd1\x9c\xd6\x93 \xca\x90:\xbev\xf4DZ-C&&0\xb8\xa7G\xe8/]'
| session_id_length= 0x20
| session_id= "Uq2)v\x1b\x88a/'\x123\x8ay\x90\x96\x11\xb5\xa3\xec_\xe7\xdd\x8e\xaf6\xa3X\xf4t\x86|"
| cipher_suite= RSA_WITH_RC4_128_SHA
| compression_method= NULL
| \extensions\
| |###[ TLS Extension ]###
| | type = 0xb00
| | length = 0x478
| |###[ Raw ]###
| | load = '\x00\x04u\x00\x04r0\x82\x04n0\x82\x03V\xa0\x03\x02\x01\x02\x02\t\x00\xd1\xe1\xf5:\x92\x03%\x1a0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x05\x05\x000\x81\x801\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x130\x11\x06\x03U\x04\x08\x13\nSome-State1\x120\x10\x06\x03U\x04\x07\x13\tSome-city1\x150\x13\x06\x03U\x04\n\x13\x0cSome-company1\x100\x0e\x06\x03U\x04\x0b\x13\x07Some-OU1\x1f0\x1d\x06\x03U\x04\x03\x13\x16some-server.some.where0\x1e\x17\r150522191612Z\x17\r250519191612Z0\x81\x801\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x130\x11\x06\x03U\x04\x08\x13\nSome-State1\x120\x10\x06\x03U\x04\x07\x13\tSome-city1\x150\x13\x06\x03U\x04\n\x13\x0cSome-company1\x100\x0e\x06\x03U\x04\x0b\x13\x07Some-OU1\x1f0\x1d\x06\x03U\x04\x03\x13\x16some-server.some.where0\x82\x01"0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x01\x05\x00\x03\x82\x01\x0f\x000\x82\x01\n\x02\x82\x01\x01\x00\xcd{qe\xeeu(\xf1\x07\xcffn\xdcg>\xed\xc8cTN\xbe\x8c\xc3t\x13F\x01^\xea\x18*s\xa9\xe1\x8e&\xf6\xf1U=\x83\x84=+\xda\xcd\xd4P\x1f\xae\xc7\xb4\xf5Dk\x87\x90\x05?\x15.#\xf7\r\x12\x1c\xa7\xf6:"\xa6WSn\xe4\xb5\x0b\x87wV\x8e\xf4i\x90\\\xe0R\x11\x17\x8d\xd9\xeb\xe2#\xb2\x12F\xcc\xe4\xba\xf3Q\xd0\xb8\x1bFH0\xe1_\xb7\x17\x8c\xf5\xf3\x9evs\xdewy\xe5\xdb\xbdz=.\xa9\x85\x89\xb0\xd6\x0065Dv\x93\xed.\xc62\xc3\xdb\xb62\xac%N;\x8c\xd7\x8e\x1e\xa1`\x98&\'\xe2\xcd:6\x9cK\xb4<HaA\xb9\x7f\xbb\xd9\xd3\xcb\x01K\x92\xe0\xecn\xcfF\xde\xd6GI\xbb\xec\xfbo\x98\xd0\xd2\xf4Y\xd5\xcf\x00T\xa6R"\x80\xaf\x96\x1d\xfc\xbe\x16P\x93q\x80\xf4=\xec\xf2\xf8r[\x94\xee\xec\x10$\x8c\xdcp\xac\xadc\xbc\xc3\xcdSp\xd0\xdc\x0f<\xba\x8d6\x99\t\xc6\xb9\x17\xf2C\xe5\xe5\xbc\'\x02\x03\x01\x00\x01\xa3\x81\xe80\x81\xe50\x1d\x06\x03U\x1d\x0e\x04\x16\x04\x14<\xc2\xf7\xfc\x85\xdb\xbeK\x05f\xb3[\xdetD\x84C\x8a\xe8>0\x81\xb5\x06\x03U\x1d#\x04\x81\xad0\x81\xaa\x80\x14<\xc2\xf7\xfc\x85\xdb\xbeK\x05f\xb3[\xdetD\x84C\x8a\xe8>\xa1\x81\x86\xa4\x81\x830\x81\x801\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x130\x11\x06\x03U\x04\x08\x13\nSome-State1\x120\x10\x06\x03U\x04\x07\x13\tSome-city1\x150\x13\x06\x03U\x04\n\x13\x0cSome-company1\x100\x0e\x06\x03U\x04\x0b\x13\x07Some-OU1\x1f0\x1d\x06\x03U\x04\x03\x13\x16some-server.some.where\x82\t\x00\xd1\xe1\xf5:\x92\x03%\x1a0\x0c\x06\x03U\x1d\x13\x04\x050\x03\x01\x01\xff0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x05\x05\x00\x03\x82\x01\x01\x00\x01s\x8e)\x85i-\x829\xfb\x17\x95\xe6\xea\x07\x18u\\\xf1\x06\xcds\x9fq\x13\xaf\xd3\xa0t\xad\xd0\x7f\x98\x1b\x06\xf3K\x9d\xf3\xe1e\x8c\x153U\xc5\x06\x1b6\x9d`\xd3A\xebL\xce\xfd\xd9\x8dmg\x90\xbeI\x9c\xde\x8b\xd5p]\x1a\x8a\x89\xbb\x14\x15\x99\xf31\x99\x14\xf8S\x9e)HH\xc1\x068b\x18\xd8g\x9d\xa4k\xa9\n,\xe7Xre\xcbU\xd6\xa6)V\x9beX\x1e\xe2\xe8\x8d\xed&K\x81\xdf\xf1\xc1\x1e,Ur\x8e\xfe\x17\r\xfeOvpo\xbb\xda\x13{\x02\xe0\xfa\x98sU\xb0\xcf\xdb?\x867\xe3Ts\xe4\xa6\xec\xcd\xcb\xc2}U\xd1\xf9V\xa5\xf2\xc4T\xe97\xdfq\xd4.!\xd4]"tw\xe2`S\xb8\xbe\x00?\xa5\'tk\x16;=K\x9aX](`\xe5\x08\x0e\xd9s}L_\xa5\xa3.\xeeE\xa4\xe5m\x8a\x03T#Ia\x90\x84X\x0c\xc9\xc6\xc2[\x1a\xc7\xf3\x85KP\x14#\xea\xfd\xd3(\x96\xaf\x92\xce\x8c\xa6\x929G\xd7|\x0e\x00\x00\x00'
Traceback (most recent call last):
File "/Users/amoneger/projects/contrib/scapy-ssl_tls/examples/full_rsa_connection_with_application_data.py", line 50, in <module>
tls_client(server)
File "/Users/amoneger/projects/contrib/scapy-ssl_tls/examples/full_rsa_connection_with_application_data.py", line 37, in tls_client
tls_client_key_exchange(sock)
File "/Users/amoneger/projects/contrib/scapy-ssl_tls/examples/full_rsa_connection_with_application_data.py", line 20, in tls_client_key_exchange
client_key_exchange = TLSRecord()/TLSHandshake()/TLSClientKeyExchange()/sock.tls_ctx.get_encrypted_pms()
File "/Users/amoneger/projects/contrib/scapy-ssl_tls/scapy_ssl_tls/ssl_tls_crypto.py", line 372, in get_encrypted_pms
raise ValueError("Cannot calculate encrypted MS. No server certificate found in connection")
ValueError: Cannot calculate encrypted MS. No server certificate found in connection
Hi,
When I ran test_ssl_tls.py and test_ssl_tls_crypto.py, I got following errors:
python test_ssl_tls.py
FAIL: test_encrypted_layer_is_decrypted_if_required (main.TestTLSDissector)
Traceback (most recent call last):
File "test_ssl_tls.py", line 175, in test_encrypted_layer_is_decrypted_if_required
self.assertTrue(server_finished_records.haslayer(tls.TLSHandshake))
AssertionError: 0 is not true
Ran 39 tests in 1.487s
FAILED (failures=1)
python test_ssl_tls_crypto.py
FAIL: test_fixed_crypto_data_matches_verify_data (main.TestTLSSessionCtx)
Traceback (most recent call last):
File "test_ssl_tls_crypto.py", line 174, in test_fixed_crypto_data_matches_verify_data
self.assertEqual(client_verify_data, binascii.hexlify(tls_ctx.get_verify_data()))
AssertionError: 'e23f73911909a86be9e93fdb' != '12003ac89553b7a233da64b9'
Ran 35 tests in 0.379s
FAILED (failures=1)
And i don't know where goes wrong?
Your sessionctx_sniffer.py examples python script, I got bellow errors when running:
Traceback (most recent call last):
File "decode_on_the_fly.py", line 110, in
sniff(iface=conf.iface, filter="tcp port 443", prn=process_ssl, store=0, count=0)
File "/usr/lib/python2.6/site-packages/scapy/sendrecv.py", line 559, in sniff
r = prn(p)
File "decode_on_the_fly.py", line 66, in process_ssl
pp = to_raw(TLSPlaintext(),session)
File "/usr/lib/python2.6/site-packages/scapy_ssl_tls/ssl_tls.py", line 811, in to_raw
crypto_container = tlsc.CryptoContainer(tls_ctx, cleartext, content_type)
File "/usr/lib/python2.6/site-packages/scapy_ssl_tls/ssl_tls_crypto.py", line 511, in init
self.hmac()
File "/usr/lib/python2.6/site-packages/scapy_ssl_tls/ssl_tls_crypto.py", line 520, in hmac
len_ = struct.pack("!H", data_len or len(self.data))
TypeError: object of type 'NoneType' has no len()
I open that file set pdb.set_trace() to print self.data, it's always is "None" so this function always return error.
The current setup file from Latest commit 03c1413 on Nov 17
Python 2.7.9 windows 10
C:\Python27\python.exe C:/Users/Shawn/Downloads/scapy-ssl_tls-master/scapy-ssl_tls-master/setup.py install
INFO: Installing scapy-ssl_tls layers to: []
Traceback (most recent call last):
File "C:/Users/Shawn/Downloads/scapy-ssl_tls-master/scapy-ssl_tls-master/setup.py", line 158, in <module>
data_files=get_layer_files_dst(get_site_packages()),
File "C:/Users/Shawn/Downloads/scapy-ssl_tls-master/scapy-ssl_tls-master/setup.py", line 66, in get_layer_files_dst
for layer_file in os.listdir(path):
WindowsError: [Error 3] The system cannot find the path specified: 'scapy_ssl_tls/*.*'
Process finished with exit code 1
making the following change to the function solves the exception though I am still hitting an error (turned out to be a env pathing issue):
def get_layer_files_dst(sites, path="scapy_ssl_tls"):
data_files = []
scapy_locations = get_scapy_locations(sites)
layer_files = []
cwd = os.path.dirname(os.path.realpath(__file__))
for layer_file in os.listdir(os.path.join(cwd,path)):
# Copy only python files, and exclude module file from copy to scapy
if layer_file != "__init__.py" and layer_file.endswith(".py"):
layer_file_path = os.path.join(path, layer_file)
if os.path.isfile(layer_file_path):
layer_files.append(layer_file_path)
for scapy_location in scapy_locations:
data_files.append(
(os.path.join(scapy_location, "layers"), layer_files))
return data_files
By default scapy-ssl_tls will not identify or dissect SSL/TLS packets on non-standard ports (i.e., != TCP 443 or TCP 4443). I found that if you manually add entries to the "bind magic" section of ssl_tls.py, you can get it to work. For example, TCP 5222 for encrypted GTalk. This is less of an issue and more of an enhancement request, perhaps the "bind magic" section should be added to ssl_tls_registry.py so that users can manually modify it?
I've only started using this tool yesterday, so I am nowhere near qualified enough to submit my own pull request for this.
When trying to resend a captured SSL-Packet I lose all data in SSL layer.
The reason seems to be some problem in the build process.
SSL('AAAAA').show()
results in
###[ SSL/TLS ]###
\records\
|###[ TLS Record ]###
| content_type= 65
| version= 0x4141
| length= 0x4141
while
SSL('AAAAA').show2()
results in
###[ SSL/TLS ]###
\records\
I tried to dig in deeper and found that in the Packet
class in the do_build
function:
def do_build(self):
if not self.explicit:
self = self.__iter__().next()
the call to self.__iter__().next()
kills the content of the SSL layer.
I don't yet understand the meaning of the explicit
attribute but setting it to 1 helps here :)
Hi again,
I want to calculate TLSFinished data using TLSSessionCtx().get_verify_data()
in 'sessionctx_sniffer' example. But i don't know where goes wrong:(
part of code: ($ means indent)
`
...
session.insert(p_ssl)
if session.crypto.session.master_secret and session.printed==False:
$print repr(session)
$session.printed = True
''' here is my adding code '''
if p.haslayer(TLSChangeCipherSpec) and p.haslayer(TLSCiphertext):
$if p.haslayer(TLSClientKeyExchange):
$$session.set_mode(client=True) #set label
$else:
$$session.set_mode(server=True) # set label
$print "---------------------------"
$print binascii.hexlify(session.get_verify_data()) #TLSFinished data (Plaintext)
$print "---------------------------"
''' end '''
print "| %-16s:%-5d => %-16s:%-5d | %s"%(p[IP].src,p[TCP].sport,p[IP].dst,p[TCP].dport,repr(p_ssl))
...
`
According to my understanding, the results should match TLSFinished data (Plaintext), in this case, which are TLSFinished(client) data='\xc2\xc7\x91Hv\x8d\xddf\xbd\xa2\xd3\xbe'(which hex is 'c2c79148768ddd66bda2d3be') and TLSFinished(server) data='1\xa9\xd7 v\r\xe1\x0e\xa4M2x'(which hex is '31a9d720760de10ea44d3278').
But my outputs are ba6317a1f96e4fe31080ba32(client, changes everytime running) and f146c4482543cb0fbff6de77(server, always be the same). And they don't match above TLSFinished data.
I've looked into get_verify_data()
and insert()
and i don't know if i misunderstand it.
I'd be very appreciate if anyone could tell me how to do it. :)
ref #77 (comment)
Btw. we can run
pycryptodome
andpycrypto
in parallel by usingpycryptodomex
package instead ofpycryptodome
. It will then have aCryptodome
namespace instead ofCrypto
. That should solve the namespace conflict.
replace requirement pycryptodome
with pycryptodomex
to avoid problems when we expect Crypto
to be pycryptodome
while it might only be pycrypto
.
setup.py
, requirements.txt
It seems like whenever I'm trying to attempt a connection with DHE_RSA_WITH_AES_128_CBC_SHA256 or DHE_RSA_WITH_AES_256_CBC_SHA256, I receive:
RuntimeError: Unsupported cipher: 0x0067 => DHE_RSA_WITH_AES_128_CBC_SHA256
Upon debugging, I observed that when get_client_kex_data()
is called, 'negotiated.key_exchange' is passed as None which causes the method to return "Key exchange unknown or currently not supported".
Any suggestions how to resolve this?
Hello,
I am trying to sniff live with this script with interface passed as an argument, but I get the error below:
Traceback (most recent call last):
File "sessionctx_sniffer.py", line 430, in <module>
main((sys.argv[1],int(sys.argv[2])), iface=iface, pcap=pcap, keyfile=keyfile)
File "sessionctx_sniffer.py", line 404, in main
sniffer.sniff(target=target, keyfile=keyfile, iface=iface)
File "sessionctx_sniffer.py", line 298, in sniff
sniff(filter="host %s and tcp port %d"%(target[0],target[1]),prn=reassemble,store=0,timeout=None)
File "/home/user/.local/lib/python2.7/site-packages/scapy/sendrecv.py", line 595, in sniff
r = prn(p)
File "sessionctx_sniffer.py", line 219, in reassemble
decrypted = self.process_ssl(p)
File "sessionctx_sniffer.py", line 179, in process_ssl
session.insert(p_ssl)
File "/home/user/.local/lib/python2.7/site-packages/scapy/layers/ssl_tls_crypto.py", line 344, in insert
self._process(p) # fill structs
File "/home/user/.local/lib/python2.7/site-packages/scapy/layers/ssl_tls_crypto.py", line 452, in _process
explicit_iv)
File "/home/user/.local/lib/python2.7/site-packages/scapy/layers/ssl_tls_crypto.py", line 823, in __init__
raise RuntimeError("Cipher 0x%04x not supported" % cipher_suite)
TypeError: %x format: a number is required, not NoneType
Any suggestions??
Currently TLSRecord length is a XLenField, which inherits from LenField. LenField calculates the length of the packet as being x = len(pkt.payload)
, which is incorrect in the cases where TLSRecords are stacked.
E.g:
stacked_pkt = tls.TLSRecord()/tls.TLSHandshake()/tls.TLSServerHello()/tls.TLSRecord()/tls.TLSHandshake()/tls.TLSCertificateList()/tls.TLSRecord()/tls.TLSHandshake()/tls.TLSServerHelloDone()
pkt = tls.TLS(str(stacked_pkt))
The length of the TLSRecord should be length until the next record is reached.
This simple unittest should pass on fix:
import unittest
import ssl_tls as tls
class TestTLSRecord(unittest.TestCase):
def test_stacked_tls_records_length_are_correct(self):
# Highlights issue https://github.com/tintinweb/scapy-ssl_tls/issues/12
# Before I try and fix it
server_hello = tls.TLSRecord()/tls.TLSHandshake()/tls.TLSServerHello()
cert_list = tls.TLSRecord()/tls.TLSHandshake()/tls.TLSCertificateList()
server_hello_done = tls.TLSRecord()/tls.TLSHandshake()/tls.TLSServerHelloDone()
stacked_pkt = server_hello/cert_list/server_hello_done
pkt = tls.TLS(str(stacked_pkt))
self.assertEqual(len(str(server_hello)) - len(tls.TLSRecord()), pkt[tls.TLSRecord].length)
I have python3 installed,I can't install scapy_ssl-tls with pip or pip3:
# pip install scapy_ssl-tls
Collecting scapy-ssl-tls
Using cached scapy-ssl_tls-1.2.2.tar.gz
Collecting scapy (from scapy-ssl-tls)
Using cached scapy-2.3.2.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-f333v30u/scapy/setup.py", line 35
os.chmod(fname,0755)
^
SyntaxError: invalid token
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-f333v30u/scapy/
Any one have any sample code for generating SKE message from server ?
you recommended using python2.x on a previous python3 support issue I created, I still can't get it to work with 2.7.
My port to python3 works without any errors but I was unable fetch the SSL/TLS layer (even through all of it shows up in pkt.show(),etc...) .
What specific python version is this working for you on? My goal is to grab the server hello,that's all I'm trying to do,if you can help me to get this working on any python version, I'll get it working with at least 3.4.
output showing the problems I'm facing:
# PYTHON=/usr/bin/python2.7 python2.7 ./examples/server_rsa.py
Traceback (most recent call last):
File "./examples/server_rsa.py", line 11, in <module>
from scapy_ssl_tls.ssl_tls import *
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 581, in <module>
class TLSServerHelloDone(PacketNoPayload):
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 583, in TLSServerHelloDone
fields_desc = [XBLenField("length", None, length_of="data", fmt="!I", numbytes=3),
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 19, in __init__
self.adjust_i2m = adjust_i2m
AttributeError: 'XBLenField' object has no attribute 'adjust_i2m'
# PYTHON=/usr/bin/python2.7 python2.7 ./examples/client_hello_rsa.py
Traceback (most recent call last):
File "./examples/client_hello_rsa.py", line 14, in <module>
from scapy_ssl_tls.ssl_tls import *
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 581, in <module>
class TLSServerHelloDone(PacketNoPayload):
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 583, in TLSServerHelloDone
fields_desc = [XBLenField("length", None, length_of="data", fmt="!I", numbytes=3),
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 19, in __init__
self.adjust_i2m = adjust_i2m
AttributeError: 'XBLenField' object has no attribute 'adjust_i2m'
# PYTHON=/usr/bin/python2.7 python2.7 ./examples/security_scanner.py
Traceback (most recent call last):
File "./examples/security_scanner.py", line 24, in <module>
from scapy_ssl_tls.ssl_tls import *
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 581, in <module>
class TLSServerHelloDone(PacketNoPayload):
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 583, in TLSServerHelloDone
fields_desc = [XBLenField("length", None, length_of="data", fmt="!I", numbytes=3),
File "/usr/lib64/python2.7/site-packages/scapy_ssl_tls/ssl_tls.py", line 19, in __init__
self.adjust_i2m = adjust_i2m
AttributeError: 'XBLenField' object has no attribute 'adjust_i2m'
Not sure how setuptools works, but it looks like the latest pip does not copy the data files to scapy. Instead it creates a new package with the name scapy_ssl_tls.
This is the result:
root@lithium:~# ls -la /opt/Python-2.7.13/lib/python2.7/site-packages/
total 1500
drwxr-xr-x 37 root root 4096 Jun 14 23:09 .
drwxr-xr-x 28 root root 20480 Jun 14 23:09 ..
drwxr-xr-x 3 root root 4096 Jun 14 23:09 asn1crypto
drwxr-xr-x 2 root root 4096 Jun 14 23:09 asn1crypto-0.22.0.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 cffi
drwxr-xr-x 2 root root 4096 Jun 14 23:09 cffi-1.10.0.dist-info
-rwxr-xr-x 1 root root 544904 Jun 14 23:09 _cffi_backend.so
drwxr-xr-x 10 root root 4096 Jun 14 23:09 Crypto
drwxr-xr-x 4 root root 4096 Jun 14 23:09 cryptography
drwxr-xr-x 2 root root 4096 Jun 14 23:09 cryptography-1.9.dist-info
-rw-r--r-- 1 root root 126 Jun 14 23:09 easy_install.py
-rw-r--r-- 1 root root 315 Jun 14 23:09 easy_install.pyc
drwxr-xr-x 2 root root 4096 Jun 14 23:09 enum
drwxr-xr-x 2 root root 4096 Jun 14 23:09 enum34-1.1.6.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 idna
drwxr-xr-x 2 root root 4096 Jun 14 23:09 idna-2.5.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 ipaddress-1.0.18.dist-info
-rw-r--r-- 1 root root 80156 Jun 14 23:09 ipaddress.py
-rw-r--r-- 1 root root 75662 Jun 14 23:09 ipaddress.pyc
drwxr-xr-x 2 root root 4096 Jun 14 23:09 .libs_cffi_backend
drwxr-xr-x 2 root root 4096 Jun 14 23:09 OpenSSL
drwxr-xr-x 3 root root 4096 Jun 14 23:09 opt
drwxr-xr-x 10 root root 4096 Jun 14 23:09 pip
drwxr-xr-x 2 root root 4096 Jun 14 23:09 pip-9.0.1.dist-info
drwxr-xr-x 4 root root 4096 Jun 14 23:09 pkg_resources
drwxr-xr-x 3 root root 4096 Jun 14 23:09 pycparser
drwxr-xr-x 2 root root 4096 Jun 14 23:09 pycparser-2.17.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 pycrypto-2.6.1.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 pyOpenSSL-17.0.0.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 readline-6.2.4.1.dist-info
-rwxr-xr-x 1 root root 578573 Jun 14 23:09 readline.so
-rw-r--r-- 1 root root 119 Jun 14 23:08 README
drwxr-xr-x 8 root root 4096 Jun 14 23:09 scapy
drwxr-xr-x 2 root root 4096 Jun 14 23:09 scapy-2.3.2.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 scapy-2.3.3.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 scapy_ssl_tls
drwxr-xr-x 2 root root 4096 Jun 14 23:09 scapy_ssl_tls-1.2.3.2.dist-info
drwxr-xr-x 4 root root 4096 Jun 14 23:09 setuptools
drwxr-xr-x 2 root root 4096 Jun 14 23:09 setuptools-36.0.1.dist-info
drwxr-xr-x 2 root root 4096 Jun 14 23:09 six-1.10.0.dist-info
-rw-r--r-- 1 root root 30098 Jun 14 23:09 six.py
-rw-r--r-- 1 root root 29563 Jun 14 23:09 six.pyc
drwxr-xr-x 2 root root 4096 Jun 14 23:09 tinyec
drwxr-xr-x 2 root root 4096 Jun 14 23:09 tinyec-0.3.1.dist-info
drwxr-xr-x 5 root root 4096 Jun 14 23:09 wheel
drwxr-xr-x 2 root root 4096 Jun 14 23:09 wheel-0.29.0.dist-info
after executing this custom location Python install:
echo "Install Python 2.7.13 ..."
# Dependencies: build-essential libssl-dev python-setuptools python2.7-dev zlib1g-dev libncurses5-dev libffi-dev
rm -r /opt/Python-2.7.13
wget --no-check-certificate https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tgz -O /tmp/Python-2.7.13.tgz
tar -zxf /tmp/Python-2.7.13.tgz -C /tmp
pushd /tmp/Python-2.7.13
./configure --prefix=/opt/Python-2.7.13 --enable-unicode=ucs4 > /dev/null && make > /dev/null && make altinstall > /dev/null
popd
rm -r /tmp/Python-2.7.13*
ln -s /opt/Python-2.7.13/bin/python2.7 /opt/Python-2.7.13/bin/python
initialPath=$PATH
initialPythonUserBase=$PYTHONUSERBASE
export PATH=/opt/Python-2.7.13/bin:$PATH
export PYTHONUSERBASE=/opt/Python-2.7.13
wget --no-check-certificate https://bootstrap.pypa.io/get-pip.py -O - | python - --user
export PYTHONUSERBASE=$initialPythonUserBase
pip install readline -t /opt/Python-2.7.13/lib/python2.7/site-packages/
pip install pyOpenSSL -t /opt/Python-2.7.13/lib/python2.7/site-packages/
pip install scapy -t /opt/Python-2.7.13/lib/python2.7/site-packages/
pip install scapy-ssl_tls -t /opt/Python-2.7.13/lib/python2.7/site-packages/ ;# setup.py for this package does not work well so we'll have to manually move files
rm /opt/Python-2.7.13/lib/python2.7/site-packages/scapy_ssl_tls/__init__*
mv /opt/Python-2.7.13/lib/python2.7/site-packages/scapy_ssl_tls/* /opt/Python-2.7.13/lib/python2.7/site-packages/scapy/layers/
rm -r /opt/Python-2.7.13/lib/python2.7/site-packages/scapy_ssl_tls*
export PATH=$initialPath
And this is what data_files=get_layer_files_dst(get_site_packages())
returns:
[('/opt/Python-2.7.13/lib/python2.7/site-packages/scapy/layers', ['scapy_ssl_tls/pkcs7.py', 'scapy_ssl_tls/ssl_tls_crypto.py', 'scapy_ssl_tls/ssl_tls.py', 'scapy_ssl_tls/ssl_tls_registry.py', 'scapy_ssl_tls/ssl_tls_automata.py'])]
I would expect the Length of the Server Hello Done to be zero, but it is three. Is there any good reason why? Do any well accepted TLS implementations have padding like this?
The problem could be with how I'm doing things. I'm trying to directly write a pcap and control the MACs, IPs explicitly. Maybe I just need a bit of advice on how best to accomplish this.
TLSv1 Record Layer: Handshake Protocol: Server Hello Done
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 7
Handshake Protocol: Server Hello Done
Handshake Type: Server Hello Done (14)
Length: 3
After installation I got:
In [1]: from scapy.all import rdpcap
File "/usr/local/lib/python2.7/site-packages/scapy/config.py", line 388
"sctp", "vrrp", "ipsec", "lltd", "vxlan", "ssl_tls" ] contribs = dict()
^
SyntaxError: invalid syntax
It seems like somewhere '\n'
is missed.
When the server returns a certificate chain, none or only the first certificate is dissected. E.g: using the full_rsa_connection_with_application_data.py against a server returning certificate chains will fail to dissect the chain:
Connected to server: ('www.some_site.com', 443)
###[ SSL/TLS ]###
\records \
|###[ TLS Record ]###
| content_type= handshake
| version = TLS_1_0
| length = 0x4a
|###[ TLS Handshake ]###
| type = server_hello
| length = 0x46
|###[ TLS Server Hello ]###
| version = TLS_1_0
| gmt_unix_time= 1433481119
| random_bytes= 'u\xd1\xe7\xe4\x8b\xaf\xa36\x089\x14\x18c\x05\xba$\x1e\xbd\xfa\xcfiW\xb8\x7f\xef\x19~\x1f'
| session_id_length= 0x20
| session_id= 'i\xb6\xda\xbb\xbd\xf7\xc1}\xb5w\xc5\x02\xdfW\xe6\xfa\xfa\x13z:\xdf\x16\x91\xe4\x02\x81\xd0\xf8\xb7\xb5h6'
| cipher_suite= RSA_WITH_RC4_128_SHA
| compression_method= NULL
| \extensions\
|###[ TLS Record ]###
| content_type= handshake
| version = TLS_1_0
| length = 0x951
|###[ TLS Handshake ]###
| type = certificate
| length = 0x94d
|###[ TLS Certificate List ]###
| length = 0x94a
| \certificates\
| |###[ TLS Certificate ]###
| | length = 0x51e
| | \data \
| | |###[ Raw ]###
| | | load = '0\x82\x05\x1a0\x82\x04\x02\xa0\x03\x02\x01\x02\x02\x02\tF0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\x000D1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x160\x14\x06\x03U\x04\n\x13\rGeoTrust Inc.1\x1d0\x1b\x06\x03U\x04\x03\x13\x14GeoTrust SSL CA - G40\x1e\x17\r150506231536Z\x17\r160311161738Z0}1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x100\x0e\x06\x03U\x04\x08\x13\x07Florida1\x180\x16\x06\x03U\x04\x07\x13\x0fFort Lauderdale1\x1c0\x1a\x06\x03U\x04\n\x13\x13Citrix Systems Inc.1\x0b0\t\x06\x03U\x04\x0b\x13\x02IT1\x170\x15\x06\x03U\x04\x03\x13\x0ewww.citrix.com0\x82\x01"0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x01\x05\x00\x03\x82\x01\x0f\x000\x82\x01\n\x02\x82\x01\x01\x00\xb5s)xd\xbdN\x053\xc9\x0b^M)\xc7\xa6qa\xe3\x04\x00\xd8\x0b\xe2\x8d\x01\xbd\xbd\x06\xe4:=\x9du;\x98q\xddk+\x97\xa8\xddy\x7f\x1e_\xee\xa9\xfeP\x87\x15\xfd\xfe\xa4\x12bU\x14\xbf\xb3\x95\xc7u\x872k\xda\x809@n\x183\xa5`\xd7}\x00\x07\xe1\xa0\xd12\x02\xda\x8eB\xfa,oZ;b\xc0\xa9}C\x97\x0fh\xb6\xe0P\xf5\xf3\xe6T\x0f\r7S\xf1\x9a\x89zK\xa3\x9b\x86\x81\xc3\\j\xfc\xf1\xcbk\xd9\xa03\x00\\s1\x97<J\x19\xa3f4d\x87\xb8\xb4@d\xa5elp%\x901\xc9N\xd6-\x0bo\x0f\x95\xbb\x9d\xc2\xfa]~\x89\xd1g\x8e\x8f:\x81}\xa4\xd5f\xd0qZ\x99\xef\xc3\x06\x15\xdb\xb1\x06\xb1$\xf6\xa5-\x0es@b\x88\xe3Y\xd9T\x84\xa0"!)\x19|\x087\x19\xca\xb2\xa5\xe8\x01\xaaYYi\xb2\xe9\xce\xe0\xe5E\x9e\xfc\x98-_X(z\xa3,\x94\rF\x1d:6\x91Z\x04\x84rCC\xa8\xcd\x02\x03\x01\x00\x01\xa3\x82\x01\xdb0\x82\x01\xd70\x1f\x06\x03U\x1d#\x04\x180\x16\x80\x14\xac2\xedZ\xc9\xe0\xde0\x9c\x90XU&c\xf6r\xa6T_\xe30W\x06\x08+\x06\x01\x05\x05\x07\x01\x01\x04K0I0\x1f\x06\x08+\x06\x01\x05\x05\x070\x01\x86\x13http://gw.symcd.com0&\x06\x08+\x06\x01\x05\x05\x070\x02\x86\x1ahttp://gw.symcb.com/gw.crt0\x0e\x06\x03U\x1d\x0f\x01\x01\xff\x04\x04\x03\x02\x05\xa00\x1d\x06\x03U\x1d%\x04\x160\x14\x06\x08+\x06\x01\x05\x05\x07\x03\x01\x06\x08+\x06\x01\x05\x05\x07\x03\x020\x81\x94\x06\x03U\x1d\x11\x04\x81\x8c0\x81\x89\x82\x15staging-cq.citrix.com\x82\x12support.citrix.com\x82\x10blogs.citrix.com\x82\x16citrixready.citrix.com\x82\x16discussions.citrix.com\x82\x0ewww.citrix.com\x82\ncitrix.com0+\x06\x03U\x1d\x1f\x04$0"0 \xa0\x1e\xa0\x1c\x86\x1ahttp://gw.symcb.com/gw.crl0\x0c\x06\x03U\x1d\x13\x01\x01\xff\x04\x020\x000Z\x06\x03U\x1d \x04S0Q0O\x06\n`\x86H\x01\x86\xf8E\x01\x0760A0?\x06\x08+\x06\x01\x05\x05\x07\x02\x01\x163https://www.geotrust.com/resources/repository/legal0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\x00\x03\x82\x01\x01\x00\x7f\xe7Yb\x14\xfdB\'\x91\x9b%\xef\xf54\xa3\xc7\x054OOu-\xb6\xa0\x0fo\x15\xff"\tz\xfd\xd260?~\x1f\xd6\x1ey\x9b4\xb1\xd5`c\xe8\x8d%\x94@n\xbe\x92\xbf}\x84\x9a\xde\xf0\xaa\x1f\xec\x13\xed;f\x8c\xd3J\xe7\xd1\x86k\xcf:\x9b\x02\xf3eV\x83\x1e\xa4\t=\xd9i\xd3\x7f\x9c\x86\x90\xc2\x81\x121r+\x82\xb6Eh\x15\x1aJ#\x87\xc1\x1b\x98\x9b\x86\xfc`\xc0o\x94\x19n\xaf\xb6<\x8d\xc6\x8eNM\xd1=V\xb2\xb7[\xe3\xb4\xf0\xa3\x14\x9d\nO/?\x89\xf3\xcf\xb0\x80f\x8a2\x1b\xf4\xe2L\x04%.\xe5\x1bs\xfb\x86\x88\xe5\xe6\x19\xfe\xe5={\xd9x\xf5\xbd$V\xe8d\x9a\xe1\xde\xea\x9b\xd9\x1a\x97\xd4\xbao\x9c&\x81\xd6\x8f\x1b\xb9\x8b\x13\xb2\x80\xdb\xf5\x87\xb2E%<S\xb5\xbaHn\x9d\xde)\x00*\xcemO?\\\xbc=\xcd$\xbf\xda;\xb6\xf0sh\x06.\xf42Sv\xe6l\xb9m\xf4\xea\xb0E\xc0\x1f\x15D\xc6\x97'
| |###[ Raw ]###
| | load = '\x00\x04&0\x82\x04"0\x82\x03\n\xa0\x03\x02\x01\x02\x02\x03\x02:y0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\x000B1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x160\x14\x06\x03U\x04\n\x13\rGeoTrust Inc.1\x1b0\x19\x06\x03U\x04\x03\x13\x12GeoTrust Global CA0\x1e\x17\r140908204110Z\x17\r220520204110Z0D1\x0b0\t\x06\x03U\x04\x06\x13\x02US1\x160\x14\x06\x03U\x04\n\x13\rGeoTrust Inc.1\x1d0\x1b\x06\x03U\x04\x03\x13\x14GeoTrust SSL CA - G40\x82\x01"0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x01\x05\x00\x03\x82\x01\x0f\x000\x82\x01\n\x02\x82\x01\x01\x00\x9a}\x98h\x11@\xc1_r\xecU\xb3\xb1c\xf32"r\x91\xc6\x16\x05\xbb\x08\x821\xb4\xf6\xee\xd4\x189\x11/.\xdaG\xfeQ1n[\xf2\xa9\n\xeb/\xbb\xf5aYeW\x02\xcd\x80\xff\xc7p2T\x89\xfd\xdb\xae\x99r\xd4O\x0c&\xb9.c0}\xde\x14[j\xd7Rx!\xf9\xbf\xbcP\xd5T\x12Y\xd8\xb56\xd9!G\xb8?jX\x1d\x8cr\xe1\x97\x95\xd3\xe1E\xa8\xf1Z\xe5\xbe\xfe\xe3S|\xa5\xf0R\xe0\xcf9\x94\x0c\x19q\xf2\xc0%\x07H}\x1c\xe6\xf19%/\x98yC\xe8\x18r\xf4e\x86\x98Z\x00\x04G\xdaKX\x1e|\x86\xb1K5\xa6 \x00\x1c\xcd\x1b;"]\xd1\x93(3\x12#\x94\x08\xaa\xc3:\xf5\xd1\xc6\x8c~\x99\xd3\x18\xa0\xad\x9d\x18\xcfI\xad\x10\x03\xf7\x993&\x86F\x9a/\xa0\xbaln\xc8\x88\x02\xb7n\xfaz\x9e\x98J\xee\x9a1}\x19\x14`\x0c\xec\x8f #<\xda\x97&\xb6\xea\x80l\x8aW\x9e \xeeo\x17%J2\xad5\x02\x03\x01\x00\x01\xa3\x82\x01\x1d0\x82\x01\x190\x1f\x06\x03U\x1d#\x04\x180\x16\x80\x14\xc0z\x98h\x8d\x89\xfb\xab\x05d\x0c\x11}\xaa}e\xb8\xca\xccN0\x1d\x06\x03U\x1d\x0e\x04\x16\x04\x14\xac2\xedZ\xc9\xe0\xde0\x9c\x90XU&c\xf6r\xa6T_\xe30\x12\x06\x03U\x1d\x13\x01\x01\xff\x04\x080\x06\x01\x01\xff\x02\x01\x000\x0e\x06\x03U\x1d\x0f\x01\x01\xff\x04\x04\x03\x02\x01\x0605\x06\x03U\x1d\x1f\x04.0,0*\xa0(\xa0&\x86$http://g.symcb.com/crls/gtglobal.crl0.\x06\x08+\x06\x01\x05\x05\x07\x01\x01\x04"0 0\x1e\x06\x08+\x06\x01\x05\x05\x070\x01\x86\x12http://g.symcd.com0L\x06\x03U\x1d \x04E0C0A\x06\n`\x86H\x01\x86\xf8E\x01\x0760301\x06\x08+\x06\x01\x05\x05\x07\x02\x01\x16%http://www.geotrust.com/resources/cps0\r\x06\t*\x86H\x86\xf7\r\x01\x01\x0b\x05\x00\x03\x82\x01\x01\x00a@\xad!\x0f\x03\xbb\x95\xdc\x89\xfc\xa3\xcb\x05q\xe9\x1cY\x975\xc2\xfak\x05\xa4\x16\xc6VF7t\x1b\x1b\xf1>,\xe87\x19\xb7\x94\xd2\x0f\x0e\xc5\xbf\x14\x07+4\xcd[\xb4\x8d\xc7V\x9d\x19\xfc\x02\xb4\x9e\x901\xfa\xa4D\xc6u\xdd\xdd\x1f%T\xa30L\xac\xdb\xfe\xc4\x88\xf71&\x18G\xaeL \x19\x1a\xc7\xae>\x98\n\x16=\xd2\xc2\xa6]\r.)}\xb2\x9d\xc7A2\x17\xca\x9d\xae9\xbf\x91\x98\xde\xe7D\xe2\x95\x9c\x94\\lB\x1bY\xc9{h\x13\xa8\x96\tt\xee@\x14\xa4\xd5\xd7\xc9{3\xa3\x0fZi\x9c\x1a\xfao\x12G\x1c\xdf\x1eLpNm\xdd\xfe\x1c\x87\xb5\x9d\xe1T\x07\t\x8a\xcd\xbe\xaa\xa8Fxn\x16\xf2\xe7\x91\x0e\xc3\xaf\xdav\x00\xd1\xd8\xa2F$\x03\xa5\x1a\x85\x81V\x83c\'\xba\x90\x8e\xf9b\x11\xba\xa7|\x90\xa9\x1af\xb4\xc5\xbc\x8f)A\xab\xeb\x8d\x99\xa6\xcc\x91d\xba\xdc\xc6\xa6L\xb3\xb4#&QrV\xf9\xf3tU\x9f%uO+'
|###[ TLS Record ]###
| content_type= handshake
| version = TLS_1_0
| length = 0x4
|###[ TLS Handshake ]###
| type = server_hello_done
| length = 0x0
Perhaps I am missing something.
https://github.com/tintinweb/scapy-ssl_tls/blob/master/examples/client_hello_rsa.py#L32
But any example with the TLSHandshakes class is not working as I cannot find it?
`
p = TLSRecord() / TLSHandshakes(handshakes=[TLSHandshake() / TLSClientHello(cipher_suites=[TLSCipherSuite.RSA_WITH_AES_128_CBC_SHA])])
`
Where is TLSHandshakes?
Hello,
When I add a SNI with length of 65000 bytes, I only see a RecordLayer but I read TLS RFC(5246), I think that they say that the maximum size of RecordLayer is 2 ^14 (16384 bytes).
Is it a bug or is it available to use bigger size than 16384?
BR
A length_from argument is passed to XBLenField, where it is not declared. Example output:
Traceback (most recent call last):
File "example.py", line 8, in <module>
from scapy.layers.ssl_tls import *
File "/usr/lib/python2.7/site-packages/scapy/layers/ssl_tls.py", line 206, in <module>
class TLSHandshake(Packet):
File "/usr/lib/python2.7/site-packages/scapy/layers/ssl_tls.py", line 209, in TLSHandshake
XBLenField("length",None, fmt="!I", numbytes=3, length_from=lambda x:x.payload),]
TypeError: __init__() got an unexpected keyword argument 'length_from'
If you want I can take a look at it tomorrow :)?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.