binarydefense / auto-ossec

Binary Defense Systems Auto-Enroll for OSSEC
Written by: David Kennedy - Binary Defense
Twitter: @HackingDave @BinaryDefense
Supported Systems: Linux, OS X, Windows

The auto ossec enrollment will automatically provision OSSEC agents for both Linux and Windows. This is accomplished through a customized protocol that interfaces with the ossec server and the automatic pairing of the server.

Descriptions:

auto_server.py - this file contains the server to house the protocol - the port needed for this is 9654. The agents will communicate with the agents to this server script that is running. This server script should be placed in an automatic init script and through watchdog in order to ensure its always running. This script needs to be placed on the OSSEC server itself.

auto_ossec.exe and auto_ossec.py - auto_ossec.exe is to run on Windows, and auto_ossec.py to run on Linux and OS X. When running the tool you will need to issue auto_ossec.exe <ip_address_of_auto_server> - this will be the IP address of the server itself.

auto_ossec.bin - precompiled for Linux with all libraries bundies in (including python-crypto)

Deployment Instructions:

Install OSSEC server on a system. Ensure that auto_server.py is automatically started when reboot occurs, and watchdog in order to ensure its always running. Also ensure no iptables conflict with port 9654 - this port is needed for the two way communication.

Install OSSEC on a Linux or Windows system as an agent. Then run auto_ossec.exe, auto_ossec.bin, or auto_ossec.py with the IP address of the SERVER that is running auto_server.py. This will automatically pair the instances of OSSEC.

NOTE THAT ALL OF THESE NEED TO BE PERFORMED WITH ROOT OR ADMINISTRATIVE LEVEL PERMISSIONS. THIS WILL FAIL IF IT IS NOT INSTALLED WITH ADMIN PRIVS.

Mass Deployment Instructions:

Create a deployment package that first installs the OSSEC binary or tar ball from (http://www.ossec.net/?page_id=19). Once the install completes, run the auto_ossec <server_ip> and you are finished. Services will automatically restart.

Ports Needed: 9654

What the server should look like when you run it in an interactive interface:

Client connected with ('192.168.170.165', 50662) [] Provisioned new key for hostname: STRONGHOLD-WIN8 with IP of: 192.168.170.165 [] Sending new key to 192.168.170.165: 8zlUouJ7yVOvt06Er8yx1zTchy5VQklfovu4SXW3GX7X8gH5tPIZ1104wvleQoZmJ9Hod++ByQtgNSLrQV7Z7rsRZLhCS9hFxPwRTZu6JC80EUXJ4yuTqFPHf9L2QuDjelP0yUvFFExf0xm7czlmDVH6/VKRdms1nL8+mwC9S81aZ0IOGpZuIMbIwiyeVxyBpctCk0Qd5CHoVZaKpAWTtA== Pairing complete. Terminating connection to client.

Linux Automatic Installation

You can now automatically install Linux as part of the auto_ossec.bin and auto_ossec.py installer. If the proper variable is specified, auto_ossec will automatically download OSSEC from the Internet, install it for you as an agent, and configure the server and keypairs for you.

In order to automatically install Linux, run the following command as root/sudo:

./auto_ossec.bin <server_ip> url=urlto/ossec.tar.gz

The server IP address is the IP address of your OSSEC server installation. The url= specifies that you want to automatically install. You can also specify a wildcard for your hostname of the AGENT. To do this you can type:

./auto_ossec.bin <server_ip> 0.0.0.0/0 url=https://bintray.com/artifact/download/ossec/ossec-hids/ossec-hids-2.8.3.tar.gz

This will automatically install OSSEC through the Internet and specify a wildcard for the IP address of the agent. This is useful when installing agents on dynamic IP addresses.

Regular Linux Installation

If you install OSSEC regularly on Linux, you can just install OSSEC normally on Linux. Be sure to specify the right server IP address of where your OSSEC server is at and the IP address of where auto_server.py is running (the server enroller).

./auto_ossec.bin <server_ip>

For the Python version, use:

pip3 install -r requirements.txt

This will install python-crypto (for AES support) and pexpect.

Then:

python auto_ossec.py <server_ip>

Install on Windows

For Windows, install OSSEC normally - since it is an MSI you should install this silently. Once OSSEC is installed, run:

auto_ossec.exe <server_ip>

This will automatically update your OSSEC config file with the server IP address and do the magic needed to pair them. You can also use a 0.0.0.0/0 (wildcard):

auto_ossec.exe <server_ip> 0.0.0.0/0

This will be useful if your system changes IP addresses frequently (dynamic DNS)

Compile on Windows (auto_ossec.py)

If you want to compile your own auto_ossec.py (instead of the auto_ossec.exe provided), follow the steps below on Windows

1. Download http://aka.ms/vcpython2 (Microsoft Visual C++ 9.0 for Python)
2. Download and Install: https://www.microsoft.com/en-us/download/details.aspx?id=5555
3. Download pyinstaller.org (latest version)
4. Download Python and Install (python.org)
5. Open up a command prompt, type: PATH=C:\Python27 (or 2 or 3 whatever)
6. python -m easy_install pycrypto
7. Unzip pyinstaller, navigate to the directory and type python pyinstaller --onefile auto_ossec.py - this will generate a binary under auto_ossec\dist

Supported Operating Systems

Linux, OS X, Windows

auto-ossec's Issues

Option to use wildcard IP addresses

Due to DHCP settings and many machines being powered on and off at different times in a retail world, we have opted to use the wildcard setting for IP addresses in the OSSEC key file. Is there any way to tell the auto_server.py script to ignore the actual address and insert a wildcard?

I have done some Python coding, but not enough to know exactly how to do this. If you could give me some pointers as to what I could change it would be great!

Thanks
Bryan Carter

32 bit windows 7

I saw in another issue that there is no support for 32 bit. Are there any other suggestions for auto deploying to 32 bit workstations. Most of my fleet are 32 bit still. We did get this working for our 64 bit workstations though, so that is great, and going forward it is great, but for the present, we are kind of back to square one.

re-register and hostid

Hi.

I have a few issues so far.
Running first time registering a Windows machine goes great, server adds it and starts to log.
The client "might" need to get reinstalled for different reasons and running auto_ossec.exe again gives me error.
Same Hostname and IP will not be accepted so I have to manually remove old entry in server, that is not ok since we need the history. It would append to the old host.

The network is restricted from using static ip in any code/script, so we need to add hostid when running auto_ossec.exe. Since the client doesn't allow hostid, the script has to resolve the hostid parameter before running script.

I would also like to be able to change aes key in server and windows binary

/Thanks

incorrect padding type error

receiving the following error when running auto_ossec.py on a linux system.
Traceback (most recent call last):
File "./auto_ossec.py", line 373, in
data = base64.b64decode(data)
File "/usr/lib64/python2.6/base64.py", line 76, in b64decode
raise TypeError(msg)
TypeError: Incorrect padding
None

Compatibility issue - 16 bit program running on 64 bit Windows

Running the auto-ossec on 64 bit Windows 7 leads to the following error.:

Even attempts to run as Windows XP or Windows 2000 gives the same result. While Python is an option, is there anyway for there to be a binary that doesn't require going that route, as it is cumbersome to deploy that at scale?

"You can't export keys on an agent"

Getting the following when I request a key from a Linux machine :-

[] The auto enrollment OSSEC Server is now listening on 9654
('Client connected with ', ('xx.xx.xx.xx', 36542))
[] Provisioned new key for hostname: ip-xx.xx.xx.xx with IP of: xx.xx.xx.xx
[] Sending new key to ip-xx.xx.xx.xx:2016/11/24 16:49:37 manage_agents: You can't export keys on an agent
Pairing complete. Terminating connection to client.
[] New OSSEC agent added - triggering restart of service to add..

Any ideas why this is happening?

Duplicate agents registered

Hi All,

I have started rolling out the agents to a large number of end points and I have noticed a little bug that could potentially cause quite a bit of work.

We deploy the agent via an automation tool. This then runs the installation at a specific time for the end points designated. What then happens is a flood of registrations coming in at a specific time, the server script then passes it on as quickly as it can process them causing duplicate entries of agent ID's. Would it be possible to make it queue connections limiting them so that the duplication is eliminated?

As a short term workaround I am scripting a sequential installation, this will make the roll out of my 4500 agents quite a bit longer.

Thanks,

Louis

auto_ossec on Windows

I cannot seem to get the Windows client to successfully register with the server. The communication occurs, but I get the below error:

c:\Program Files (x86)\BinaryDefense\AutoOSSEC>auto_ossec.exe serverip *
[*] Connected to auto enrollment server at IP: serverip
[*] Pulled hostname and IP, encrypted data, and now sending to server.
[*] We received our new pairing key for OSSEC, closing server connection.
[*] Removing any old keys.
[*] Successfully imported the new pairing key.
[*] Stopping the OSSEC service, just in case its running.
[!] Unable to find the ossec.conf file in: /var/ossec/\ossec.conf
[!] Please install OSSEC first before running any of this.

Note: my OSSEC server is an AlienVault USM. Not sure if the above error message is related to the client or the server, but on the server side I believe the ossec.conf is located in /var/ossec/etc.

Any suggestions would be really appreciated. Auto-ossec seems like a wonderful solution to the pain of mass deployment.

agent not being registered in server

Hi I am using ossec server 2.8.3 on a centos 7.3 machine and client side a Windows 2012 R2 machine with ossec agent 2.9.2. When running auto-ossec <server_ip>, I get the following output;

Python version 3.5

Client side:
[] Connected to auto enrollment server at IP: 10.18.119.14
[] Pulled hostname and IP, encrypted data, and now sending to server.
[] We received our new pairing key for OSSEC, closing server connection.
[] Removing any old keys.
[] Successfully imported the new pairing key.
[] Stopping the OSSEC service, just in case its running.
[] Overwriting the ossec.conf to incorporate server host IP address.
[] Finished. Started the OSSEC service. Auto Enrollment for OSSEC is now finish

Server side:
Client connected with ('10.18.139.101', 58745)
new() missing 1 required positional argument: 'mode'
Traceback (most recent call last):
File "auto_server.py", line 164, in handle
data = aescall(secret, data, "decrypt")
File "auto_server.py", line 145, in aescall
cipher = AES.new(secret)
TypeError: new() missing 1 required positional argument: 'mode'
Pairing complete. Terminating connection to client.

But the agent is not being registered and hence no key can be retrieved and added to the agent config. Can you please help?

Does this tool support re-enrollment?

With tools like ossec-authd you can automatically register agents to the ossec server but there is no way to see if a host is already registered. Does this tool support re-registering hosts with the same key/name/ip?

Auto restart of OSSEC server service upon agent registration appears to be broken

Procedure ossec_monitor() in auto_server.py appears intended to check every 5 minutes to see if any auto-registrations have occurred and if so to restart the OSSEC server services. What appears to actually happen is the 5 minute timer is started when auto_server.py is first run, and at the end of 5 minutes it does a one-time check for the auto-registration flag file and restarts OSSEC if the flag file is there. It never checks again after that. I put a "while true" loop around the contents of ossec_monitor() and then it kept on checking every 5 minutes, but it had a maddening side-effect of the tcp/9654 socket getting handed off to the OSSEC analysisd daemon upon stopping auto_server.py, making auto_server.py unable to start until OSSEC was restarted first. I ended up using a cron job to do the work of ossec_monitor independently from auto_server.py.

server script fails taking ossec down

Hi,

I run the script permanently with a scheduled restart of the script and OSSEC at 6AM every morning (not the vm just the services)

I have now found this in my logs for every restart instance. Any help would be appreciated.

SERVICE RESTART COMPLETE
Sun May 8 06:00:05 SAST 2016

[*] The auto enrollment OSSEC Server is now listening on 9654
Traceback (most recent call last):
File "/var/ossec/bin/new_auto_ossec", line 171, in
t = ThreadedTCPServer(('',9654), service)
File "/usr/lib64/python2.6/SocketServer.py", line 412, in init
self.server_bind()
File "/usr/lib64/python2.6/SocketServer.py", line 423, in server_bind
self.socket.bind(self.server_address)
File "", line 1, in bind
socket.error: [Errno 98] Address already in use

Seems it is not happy with the command " t = ThreadedTCPServer(('',9654), service)"

32-Bit Compatibility Issue - Windows 64-Bit

I can't run the .exe on a 64-bit windows server.

Is this a known bug?

auto_server.py python 3.x only

This seems to be Python 3.x compatible only
Had to change 'socketserver' to SocketServer to able to support Python 2.x

/C

Mac support

You might want to consider adding support for Mac OSSEC agents. I found I was able to get this to work by making auto-ossec.py treat a Mac just like you treat Linux OSSEC agents. That, plus use "/var/ossec/bin/ossec-control stop/start" to control the OSSEC agent service rather than "service ossec stop/start". That works for both Mac and Linux platforms. I've only tested Mac support with a Mac running El Capitan. You can see how I did it here:

https://github.com/branchnetconsulting/auto-ossec/blob/master/auto_ossec.py

By the way, at least with Mac OS "El Capitan", I was pleased to find that the OSSEC agent installs from tarball very cleanly now. You just need pycrypto and Xcode installed. Older versions of Mac OS and OSSEC required special tweaks to get OSSEC agent running on a Mac. The only extra step I had to take was this one to make OSSEC agent start at boot time.

https://wikis.utexas.edu/display/ISO/Configuring+OSSEC+HIDS+on+OS+X+Yosemite#ConfiguringOSSECHIDSonOSXYosemite-Automaticallylaunchatsystemboot

Thanks for your great work on this,
Kevin

suddenly i cannot use auto_ossec anymore..

Dear,

i was using auto_ossec before without problem, and all was perfect.
Now, i reinstalled security onion and cannot add anymore new clients (tried on two instance and it is the same behavior).

On windows side after run i got:
auto_ossec.exe X.X.X.X [*] Connected to auto enrollment server at IP: X.X.X.X [*] Pulled hostname and IP, encrypted data, and now sending to server. [*] We received our new pairing key for OSSEC, closing server connection. [*] Something did not complete. Does this system have Internet access? Traceback (most recent call last): File "auto_ossec.py", line 369, in <module> NameError: name 'path' is not defined None

on linux/server side:
`root@so01:~/auto-ossec# python3 auto_server.py
[*] The auto enrollment OSSEC Server is now listening on 9654
Client connected with ('X.X.X.X', 50928)
Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7fee78057dd8>
command: /var/ossec/bin/manage_agents
args: ['/var/ossec/bin/manage_agents']
searcher: None
buffer (last 100 chars): b': X.X.X.X\r\nConfirm adding it?(y/n): '
before (last 100 chars): b': X.X.X.X\r\nConfirm adding it?(y/n): '
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 26833
child_fd: 8
closed: False
timeout: 300
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 97, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 452, in read_nonblocking
raise TIMEOUT('Timeout exceeded.')
pexpect.exceptions.TIMEOUT: Timeout exceeded.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "auto_server.py", line 224, in handle
provision_key(hostname, ipaddr)
File "auto_server.py", line 197, in provision_key
ossec_key = parse_client(hostname, ipaddr)
File "auto_server.py", line 81, in parse_client
child.expect("for the new agent")
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 315, in expect
timeout, searchwindowsize, async)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 339, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 104, in expect_loop
return self.timeout(e)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 68, in timeout
raise TIMEOUT(msg)
pexpect.exceptions.TIMEOUT: Timeout exceeded.
<pexpect.pty_spawn.spawn object at 0x7fee78057dd8>
command: /var/ossec/bin/manage_agents
args: ['/var/ossec/bin/manage_agents']
searcher: None
buffer (last 100 chars): b': X.X.X.X\r\nConfirm adding it?(y/n): '
before (last 100 chars): b': X.X.X.X\r\nConfirm adding it?(y/n): '
after: <class 'pexpect.exceptions.TIMEOUT'>
match: None
match_index: None
exitstatus: None
flag_eof: False
pid: 26833
child_fd: 8
closed: False
timeout: 300
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
Pairing complete. Terminating connection to client.

is something with new version of security onion or windows (win10 1803)?

tnx.

Something Did Not Complete: Name 'path' is not defined

Hello,

so I THINK I setup auto_ossec correctly. I have a windows 10 system, and the OSSEC appliance on a VM. But when I run the auto_ossec.exe file on my windows system, it grabs the key then claims something didn't complete. Then it prints "name 'path' is not defined".

Any help? Also, on my server I put auto_ossec.py into chkconfig in /etc/init.d/ is this correct? When I invoke service auto-ossec (no matter the flag after) It prints this:

service auto-ossec status
[*] The auto enrollment OSSEC Server is now listening on 9654
Traceback (most recent call last):
File "/etc/init.d/auto-ossec", line 176, in
t = ThreadedTCPServer(('',9654), service)
File "/usr/lib64/python2.6/SocketServer.py", line 412, in init
self.server_bind()
File "/usr/lib64/python2.6/SocketServer.py", line 423, in server_bind
self.socket.bind(self.server_address)
File "", line 1, in bind
socket.error: [Errno 98] Address already in use

Which I take to mean the auto-ossec part of it is running? Ossec is running too, but it doesn't update anything.

Any help is appreciated. I know I did something wrong, but I'm pulling my hair out trying to figure out what!

auto_server.py times out if /etc/client.keys has large number of entries

If I manually open manage_agents and add an agent I see the CPU jumps to 100% and it takes anywhere from 20-60 seconds to add a new agent. This is due to having 1,417 entries in my client.keys. Sometimes when the delay is too large auto_server.py will timeout though it says the key was created ok. I've managed to work around it by adding a virtual CPU to soak up the load however at some point to number of entries will grow to the point where it will happen again. I was wondering if their is a way to adjust the script to increase the timeout when it's waiting for manage_agents to respond?