kamushadenes / cefevent

ArcSight's Common Event Format library

Henrique Espíndola

Security Specialist and Entrepreneur

Co-Founder and CTO @ Stellar Entropy

Leadership | Security | DevOps | Programming | Networking | Cloud | Crypto

https://blog.hadenes.io

About Me

Hello and welcome!

My name is Henrique, and I'm an experienced security professional with expertise in risk assessment and management, security engineering, pentesting, reversing, incident response, and compliance. I've developed and implemented complex infrastructures for fast-growing startups in the financial services industry, and have a proven track record of delivering results.

In addition to my security expertise, I'm proficient in programming languages such as Python, Golang, Clojure, and Rust, and have a demonstrated ability to develop and maintain large-scale software systems. I'm passionate about staying up-to-date on the latest threats and techniques in the industry, and enjoy pursuing DIY projects related to electronics and programming.

If you're looking for a security professional with both technical and management skills, I have a wealth of experience leading and managing cross-functional teams of over 30 professionals.

Thank you for stopping by, and I look forward to connecting with you!

Noteworthy Public Projects

SQmaiL allows you to query your IMAP email server using SQL.

Chloe is a powerful AI Assistant written in Go that leverages OpenAI technologies (ChatGPT, Whisper, and DALL-E) along with Google's Text-to-Speech engine to provide versatile and comprehensive assistance.

It offers multiple interfaces and utilizes the Chain of Thought approach to understand and respond to complex instructions.

Cryptographically attest if your Go application is being executed within a cloud environment.

APKrash is an Android APK security analysis toolkit focused on comparing APKs to detect tampering and repackaging.

Generate, validate and send CEF events.

Script suite to parse financial transactions from brazilian bank and benefit accounts, including support (when applicable) for interest fees and overdraft limits, besides account metadata and account holder information, when available. Now deprecated, please use https://quan.to.

Work and Life

Work

Hobbies and Passions

Game Platforms

Games

Skills

Languages

IDEs

Cloud Providers

Networking

Databases

Data

Observability

Virtualization, Containers and Orchestration

Automation and Infrastructure as Code

Hardware and IoT

CI/CD

Frameworks and Tools

Operating Systems and Distros

Stats

cefevent's Issues

Rewrite CEF Event

This is an old project, in need of some refactoring.

What do you folks think about having it rewritten in Go?

My arguments:

• Easier deployment in restricted environments (only a single binary), as was usually the case when I consulted ArcSight
• Better data assertion with types
• Better (although uglier) error handling
• Likely a higher EPS thoughput with the use of goroutines and channels
• I like Go :D

@tristanlatr
@pcktdmp
@urbantom
@lewisoaten

flexString fields

Hi Kamus, thanks for this wonderful classs.
I noticed an Issue on flex fields i saw is that flexString1 should be decoded to "flexString1" in CEF and not to "fs1".
maybe it used to be like this in the past but now it not working.

Add some examples of example_cef_csv.csv

Hello!

I do not have any awesome examples to create csv file with right headers for input to your programm.
Can you add to project 1-2 csv files with examples?

Big thx.

duplicate entries in _extension_dictionary dictionary

cefevent/cefevent/__init__.py

Line 87 in d6c60f0

'dhost': {'full_name': 'DestinationHostName', 'data_type': 'String', 'length': 1023, 'description': 'Identifies the source that an event refers to in an IP network. The format should be a fully qualified domain name associated with the "destination node, when a node is available. Examples: "host.domain.com" or "host".'},

Same for 'request' and 'spriv' tags

CN fullname repeated

cefevent/cefevent/__init__.py

Lines 48 to 53 in d6c60f0

	'cn1': {'full_name': 'deviceCustomNumber1', 'data_type': 'Long', 'length': 0, 'description': 'There are three number fields available which can be used to map fields which do not fit into any other field of this dictionary. If possible, "these fields should not be used, but a more specific field from the dictionary. Also check the guidelines hereafter for hints on how to utilize these fields.'},
	'cn1Label': {'full_name': 'deviceCustomNumber1Label', 'data_type': 'String', 'length': 1023, 'description': 'All custom fields have a corresponding label field where the field itself can be described. Each of the fields is a string describing the purpose of this field.'},
	'cn2': {'full_name': 'deviceCustomNumber1', 'data_type': 'Long', 'length': 0, 'description': 'There are three number fields available which can be used to map fields which do not fit into any other field of this dictionary. If possible, "these fields should not be used, but a more specific field from the dictionary. Also check the guidelines hereafter for hints on how to utilize these fields.'},
	'cn2Label': {'full_name': 'deviceCustomNumber2Label', 'data_type': 'String', 'length': 1023, 'description': 'All custom fields have a corresponding label field where the field itself can be described. Each of the fields is a string describing the purpose of this field.'},
	'cn3': {'full_name': 'deviceCustomNumber1', 'data_type': 'Long', 'length': 0, 'description': 'There are three number fields available which can be used to map fields which do not fit into any other field of this dictionary. If possible, "these fields should not be used, but a more specific field from the dictionary. Also check the guidelines hereafter for hints on how to utilize these fields.'},
	'cn3Label': {'full_name': 'deviceCustomNumber3Label', 'data_type': 'String', 'length': 1023, 'description': 'All custom fields have a corresponding label field where the field itself can be described. Each of the fields is a string describing the purpose of this field.'},

Looks like a copy and paste error with the fullName field.

Similar to #1 .

issues with running module / run.py

I get this error when running run.py. File /con/CEFtools/logfiles/test_out.log contains a CSV file with CEF header.
It seems the module struggles finding his own imports. Same on Linux or Windows.

[cefevent]# /opt/rh/rh-python36/root/bin/python ./run.py --host localhost --port 10514 --auto_send --eps 10000 /con/CEFtools/logfiles/test_out.log
Traceback (most recent call last):
File "./run.py", line 2, in
from cefevent import CEFSender, CEFEvent
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/cefevent/cefevent.py", line 4, in
from .extensions import extension_dictionary
ImportError: attempted relative import with no known parent package

Ok, so I remove the . in .extensions to have it find it, we go one step further

[cefevent]# /opt/rh/rh-python36/root/bin/python ./run.py --host localhost --port 10514 --auto_send --eps 10000 /con/CEFtools/logfiles/test_out.log
Traceback (most recent call last):
File "./run.py", line 2, in
from cefevent import CEFSender, CEFEvent
ImportError: cannot import name 'CEFSender'

So I modify the import line to
from cefsender import CEFSender

And I get this:

[cefevent]# /opt/rh/rh-python36/root/bin/python ./run.py --host localhost --port 10514 --auto_send --eps 10000 /con/CEFtools/logfiles/test_out.log
Traceback (most recent call last):
File "./run.py", line 4, in
from cefsender import CEFSender
File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/cefevent/cefsender.py", line 6, in
from cefevent.syslog import Syslog
ModuleNotFoundError: No module named 'cefevent.syslog'; 'cefevent' is not a package

All righty, trying with modifying that line in cefsender.py to
from cefevent import CEFEvent

I don’t think it requires all this tweaking, may be something I’m missing.

Also, it seems to resend the same events over and over. How can I make it only send the content of the file once?
And there seem to be a few fields missing from extensions.py (deviceVendor, deviceProduct, Name, deviceEventClassId, requestContext etc). It would be good to add them. Please refer to the list sent to you by email.

Thanks

Can't set field value to 0

event = CEFEvent()
event.set_field('cn1', 0)
string = event.build_cef()
assert 'cn1=0' in string

its expected to result 'cn1=0' in the string.

the fix is:

def set_field(self, field, value):

        if field in self._prefix_list:
            return self.set_prefix(field, value)

        if field in self._reverse_extension_dictionary:
            v = self._validate_field_value(field, value)
            if v:  <------- THIS SHOULD BE `if v is not False:`!!
                self.extensions[field] = v
                return self.extensions[field]
            else:
                return False
        elif field in self._extension_dictionary:
            field = self._extension_dictionary[field]['full_name']
            v = self._validate_field_value(field, value)
            if v:
                self.extensions[field] = v
                return self.extensions[field]
            else:
                return False

bytesIn/bytesOut

Hey @kamushadenes - I just stumbled across this valuable project, really appreciate your work.
I recently figured out, that in the current CEF definition, bytesIn and bytesOut is allowed to be Long from CEF version 1.0 onwards
(see V26 of CEF implementation guide)

do you have any thoughts of implementing these two different standards?
Cheers
A

Error handling based on for instance ValueError instead of returning False

Currently during build_cef() when a constraint barrier is hit in most cases False is being returned.
I would like to propose to use raise so these errors can be catched and handled.
A simple example is for instance the field length of requestClientApplication.

TCP Does not work

Hi,

I am trying to send Syslog events to a local Syslog server. For UDP, my messages work find, however, with TCP, I do not see any messages coming to the Syslog.

I noticed that the data is encoded to 'utf-8' for TCP. Is this the right encoding for TCP? Does TCP work for you?

Thanks,
Steve

Data CEF

Hi all,

Could you tell me if you introduce random data in your generator CEF ?

Thx