dreadl0ck / ja3

Go package for Ja3 TLS client and server hello fingerprints

ja3's Issues

Normalizing Client Hellos

Is there a mechanism to normalize client hellos? Chrome is randomizing order: https://hnull.org/2022/12/01/sorting-out-randomized-tls-fingerprints/

Invalid Ja3 when GREASE value is at the end of extensions

ja3/ja3.go

Lines 121 to 139 in 0c3c870

	// collect extensions
	lastElem = len(hello.AllExtensions) - 1
	if len(hello.AllExtensions) > 1 {
	for _, e := range hello.AllExtensions[:lastElem] {
	// filter GREASE values
	if !greaseValues[uint16(e)] {
	buffer = strconv.AppendInt(buffer, int64(e), 10)
	buffer = append(buffer, sepValueByte)
	}
	}
	}
	// append last element if extensions are not empty
	if lastElem != -1 {
	// filter GREASE values
	if !greaseValues[uint16(hello.AllExtensions[lastElem])] {
	buffer = strconv.AppendInt(buffer, int64(hello.AllExtensions[lastElem]), 10)
	}
	}
	buffer = append(buffer, sepFieldByte)

I'm seeing a small discrepancy between this library and pyja3. When the final extension is a GREASE value, a trailing '-' is left on. It appears this can happen with ciphers as well, but I haven't seen that actually manifest.

Example:

771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,23-18-51-43-16-10-65281-17513-13-27-5-11-35-45-0,29-23-24,0 (pyja3)
771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,23-18-51-43-16-10-65281-17513-13-27-5-11-35-45-0-,29-23-24,0 (this library)

Here's a sample hex client hello to reproduce:

1603010201010001fd030316aa71dc4b3931d23f02de5b79220115f7c0148b91b0b2f2ad580dae1437c927206f244e8429100fdfb4501431c6c354d80e2b284251995d8f4f8bfe7ac7402ccd00205a5a130113021303c02bc02fc02cc030cca9cca8c013c014009c009d002f0035010001944a4a000000170000001200000033002b0029fafa000100001d00201929be474921dfe2125cdeca802e4e0610e76ff2ca4ab27e57d7a182a58fd205002b0007060a0a030403030010000e000c02683208687474702f312e31000a000a0008fafa001d00170018ff01000100446900050003026832000d0012001004030804040105030805050108060601001b0003020002000500050100000000000b00020100002300d082491ebd07c9757ca3daa0d7612b23d3edf38e075325a044ac4ac89eae5ab05b1ead8f97b87d3fde5774bb373d2d7a92c615ebbd3ec50b71dcfc465459295843dceae0ea377e592f355f22d5f32f27ff051daf8be16e8c2768bf1f8dba24f44258b9fc3f5872d4e7fd791171361b05f3ed2aca177c3a6e69ae7452af76d9176fd8538a8c364af5b37d0f6de6af0f065632e603d9b9c04d5fc49701bf5bf6bafce840bd0de75acc00a8a2d06b764a08a485f9ebb642d69c834f6e4319fc7cf584e59c9b04942b8d1c78feb876717e43b3002d0002010100000011000f00000c69622e61646e78732e636f6dfafa000100

flag provided but not defined: -encodedCommand

I am getting error, when I am using this ja3 library on windows for live capture analysis

Proposal: Please start using Semantic Versioning

I found that this project already supports Go modules. But sadly, the tags doesn't follow Semantic Versioning, which means that all tags of this project will be ignored by Go modules and replaced by pseudo-versions, go get acts weirdly when tags are not in that form. It would be great to have the tagged release be named in the format vX.X.X format so that go mod can read it.

$go get github.com/dreadl0ck/[email protected]
go get github.com/dreadl0ck/[email protected]: no matching versions for query "v1.0"
$go get github.com/dreadl0ck/ja3
go: downloading github.com/dreadl0ck/ja3 v0.0.0-20200304222813-39f4a089559b
go: github.com/dreadl0ck/ja3 upgrade => v0.0.0-20200304222813-39f4a089559b

Else the mod file shows something like github.com/dreadl0ck/ja3 v0.0.0-20200304222813-39f4a089559b which is not very readable and difficult to upgrade. It’s hard to verify which version is in use. This is not conducive to version control.

So, I propose this project to follow Semantic Versioning in future versions. For example, v1.0.1, v2.0.0, v3.1.0-alpha, v3.1.0-beta.2etc, so that other project can use tag in go.mod.

how to generate ja3 of incoming requests on go server?

Hello, great library, i am trying to use it to generate fingerprints of incoming requests to my go server to enhance security, is there any way to pass the request to a method and get back the ja3? i ve seen the documentation and it accepts gopacket.Packet or *tlsx.ServerHelloBasic but cant understand how i am supposed to extract them from the request got in my server. Thx for the support.

(p.s. it might be helpful to have basic usage examples in the doc, i dont think there is unless i ve missed it )

Error on running goja3 on win machine

goja3.exe -iface Ethernet0
panic: Loopback: No such device exists (No such device exists)

goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0xc0000100b8, 0x8, 0xcbf000, 0xc000006018, 0xc718f6, 0x1, 0xc000070101, 0x5ea, 0xc00003e001, 0xffffffffff676980)
C:/Users/win/Desktop/ja3-master/live.go:23 +0x26e5
main.main()
C:/Users/win/Desktop/ja3-master/cmd/main.go:56 +0x34e

Machine is a VM and have Ethernet0

Idx Met MTU State Name

---

12 25 1500 connected Ethernet0

use of dreadl0ck/gopacket produces duplicate symbols

Commit 3e0281e
broke my code. I use google/gopacket for my own analysis, and dreadl0ck/gopacket is producing conflicts.

When I try and run my code, I see

duplicate symbol '_pcap_wait' in:
    /var/folders/lt/fwtf8x396cbbd39911cdfrl400724v/T/go-link-974407391/000019.o
    /var/folders/lt/fwtf8x396cbbd39911cdfrl400724v/T/go-link-974407391/000021.o
duplicate symbol '_pcap_next_ex_escaping' in:
    /var/folders/lt/fwtf8x396cbbd39911cdfrl400724v/T/go-link-974407391/000019.o
    /var/folders/lt/fwtf8x396cbbd39911cdfrl400724v/T/go-link-974407391/000021.o
duplicate symbol '_pcap_offline_filter_escaping' in:
    /var/folders/lt/fwtf8x396cbbd39911cdfrl400724v/T/go-link-974407391/000019.o
    /var/folders/lt/fwtf8x396cbbd39911cdfrl400724v/T/go-link-974407391/000021.o
ld: 3 duplicate symbols for architecture x86_64

duplicate symbols:

grep -r _pcap_wait *
Binary file pkg/darwin_amd64/github.com/google/gopacket/pcap.a matches
Binary file pkg/darwin_amd64/github.com/dreadl0ck/gopacket/pcap.a matches

objdump -syms pkg/darwin_amd64/github.com/dreadl0ck/gopacket/pcap.a  |grep pcap_wait
0000000000000bb0 g     F __TEXT,__text	__cgo_216e6da8ebe5_Cfunc_pcap_wait
0000000000000030 g     F __TEXT,__text	_pcap_wait


objdump -syms pkg/darwin_amd64/github.com/google/gopacket/pcap.a   |grep pcap_wait
0000000000000bb0 g     F __TEXT,__text	__cgo_193fe916b310_Cfunc_pcap_wait
0000000000000030 g     F __TEXT,__text	_pcap_wait

I'm trying to use dreadl0ck/ja3 as an import in my own code.

request for pcap.h

When I try to compile ja3 I am prompted with the following error

go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
# github.com/google/gopacket/pcap
../test/pkg/mod/github.com/google/[email protected]/pcap/pcap_unix.go:34:18: fatal error: pcap.h: No such file or directory
compilation terminated.

runtime error: slice bounds out of range

when I run this /root/go/bin/goja3 -ja3s=false -json -iface eth1

the result

[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth1
timestamp,source_ip,source_port,destination_ip,destination_port,ja3_digest
1600315798.710436,128.59.66.11,443,10.246.120.106,61720,dd4b012f7a008e741554bd0a4ed12920
1600315798.710710,61.129.7.21,443,10.246.52.102,62627,389ed42c02ebecc32e73aa31def07e14
1600315798.713353,61.155.222.97,443,10.246.121.186,53185,3653a20186a5b490426131a611e01992
1600315798.717188,61.155.222.97,443,10.246.121.186,53186,3653a20186a5b490426131a611e01992
1600315798.718566,10.246.92.133,2301,180.101.212.39,443,3b5074b1b5d032e5620f69f9f700ff0e
1600315798.732672,180.101.212.39,443,10.246.120.138,53336,e9a3ced07403903421d84ae7e9d92be5
1600315798.735224,10.246.142.18,41458,172.217.160.110,443,66918128f1b9b03303d77c6f2eefd128
1600315798.735948,10.246.142.18,41460,172.217.160.110,443,66918128f1b9b03303d77c6f2eefd128
1600315798.738636,10.246.152.123,43522,118.193.98.74,443,9b02ebd3a43b62d825e1ac605b621dc8
1600315798.739077,10.246.104.50,2608,180.163.32.196,443,44d502d471cfdb99c59bdfb0f220e5a8
1600315798.741837,192.168.104.159,61602,116.211.20.187,8514,455bd65d382d4741f0e48654f27cbe80
1600315798.742983,10.246.40.103,49517,117.18.232.200,443,3b5074b1b5d032e5620f69f9f700ff0e
1600315798.747841,119.23.53.168,443,10.246.92.224,61554,4ef1b297bb817d8212165a86308bac5f
1600315798.749056,10.246.152.180,38806,157.255.245.29,443,c4d9c7fc50e1fd2ebf09e3d9a2a31b33
1600315798.749270,119.23.53.168,443,10.246.92.224,61555,4ef1b297bb817d8212165a86308bac5f
1600315798.752894,139.227.253.126,6690,10.246.34.52,6672,20e0bd1c4d36e08a1f31656fb48b99a0
1600315798.756333,10.246.114.99,54261,172.217.160.74,443,b32309a26951912be7dba376398abc3b
1600315798.756440,61.174.240.228,443,10.246.138.147,39796,4cf820cab8f5a2bf61be14f5493233ae
panic: runtime error: slice bounds out of range [:15] with capacity 12

goroutine 1 [running]:
github.com/dreadl0ck/tlsx.(*ServerHelloBasic).Unmarshal(0xc00015d830, 0xc000356c6a, 0xc, 0xc, 0x4c984a, 0x0)
	/root/go/pkg/mod/github.com/dreadl0ck/[email protected]/serverHello.go:270 +0x7d5
github.com/dreadl0ck/ja3.BarePacketJa3s(0x63b980, 0xc00037cdc0, 0xee0e38, 0x0, 0x0)
	/home/morty/ja3/gopacket.go:119 +0x110
github.com/dreadl0ck/ja3.DigestHexPacketJa3s(0x63b980, 0xc00037cdc0, 0x0, 0x0)
	/home/morty/ja3/gopacket.go:58 +0x39
github.com/dreadl0ck/ja3.ReadInterfaceCSV(0x7ffc2e4667e9, 0x4, 0x6360e0, 0xc000010018, 0x5e9566, 0x1)
	/home/morty/ja3/live.go:51 +0x1846
main.main()
	/home/morty/ja3/cmd/main.go:44 +0x2e2

Missing ignore for GREASE values in ja3 digest

Hello!

JA3 are to ignore values utilizing GREASE (https://tools.ietf.org/html/draft-davidben-tls-grease-01).

As given by https://github.com/salesforce/ja3/blob/master/python/ja3.py all values: ciphers, extensions, elliptic curve and elliptic curve point formats, should be filtered so GREASE values are not added to the ja3 digest. The ja3 authors probably did not mean to filter ecpf in GREASE since its a uint8.

Example on how this is done can be found here:
https://github.com/D4-project/sensor-d4-tls-fingerprinting
(Tho, that library is missing filtering GREASE for elliptic curves, see D4-project/sensor-d4-tls-fingerprinting#13)

If this is not done you will not produce the correct hash.

Cheers,
Kjell Tore Fossbakk

Upgrade utls to support application extension for ja3

@dreadl0ck Can you help to upgrade for TLS application extension?

Reference:
refraction-networking/utls#113
https://github.com/refraction-networking/utls/releases/tag/v1.1.2