pjlantz / hale Goto Github PK

Implement automated file submission to sandbox for analysis

Add more visualization

Add CSRF protection to work together with the oauth post method

Refactor the main code into a client/server architecture. Communication between these will be done with the perspective broker system in twisted (remote method calls).

Create documentation for the monitor, user and development docs at github wiki

Feature to add bot that support the dionaea xmpp server

Should take care of configurations for the different modules. Since they will vary it should
check if current config is correct before executing a module and notify the user if thats not the case.

Sensors should update their modules themself from the db instead of the user having to download it from the web ui and drag it to the modules directory.

Implement install procedure

Implement logging component to put collected logs from the various modules into the db and to the xmpp share channel

This component offers users access to the botnet logs, info and captured files. Also there will be a feature to upload modules and add different configuration depending on module. Adding proxy servers and their credentials should also be possible.

Visualization will be done of the data stored in the db with different charts and maps to better present this information.

Search function implemented with Haystack.

Restful Web API with Django-Piston.

Design a database that should basically hold modules and different configurations for that module and user comments. Also store logs, captured files and info from tracked botnets and a list of proxies and their credentials.

South will be used for schema migration and django as a db api. Implementation will try to be database independent.

Implementation of a http-based monitor

Fix the socks5 package to handle disconnection in a smooth way. When executing a module, a proxy server should be chosen based on a round-robin algorithm. Proxy server and credentials should be fetched from the central db.

Module to perform sending of files for analysis to a sandbox. Submissions made to mwanalysis or to anubis depending on if access is given to the backend. Result/Analysis URL stored in central db.

Design and implement a dynamic module loading system. This will basically be used to load new modules, as for the first stable version irc and http modules are planned to be implemented. Some poc code is listed at http://gist.github.com/422458

django-piston OAuth support for web API

Implementation of a irc-based module and some sort of a thread manager to handle starting and stopping module threads.

When I run manage.py with sqlite3 in settings.py I get error when running syncdb for first time

(env) a@a-virtual-machine ~/irc/Hale/src/webdb $ python manage.py syncdb
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_manager(settings)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 209, in execute
    translation.activate('en-us')
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 66, in activate
    return real_activate(language)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/utils/functional.py", line 55, in _curried
    return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/utils/translation/__init__.py", line 36, in delayed_loader
    return getattr(trans, real_name)(*args, **kwargs)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 193, in activate
    _active[currentThread()] = translation(language)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 176, in translation
    default_translation = _fetch(settings.LANGUAGE_CODE)
  File "/home/a/irc/Hale/env/local/lib/python2.7/site-packages/django/utils/translation/trans_real.py", line 160, in _fetch
    apppath = os.path.join(os.path.dirname(app.__file__), 'locale')
AttributeError: 'module' object has no attribute '__file__'

When required, it should be possible to start a centralized node that takes care of all botnet logs, info and captured files and put this stuff in the database. May also handle malware analysis and authentication to this type of server. Since the information can grow its better for one node to take care of this instead of several ones.

This should basically be a server that offers remote method calls and put everything in a queue before its inserted to the database. Planned to be implemented with twisted perspecetive broker. First version before the logging component is implemented and db is designed is to receive data from clients and output it to the screen.

Update search index direct on model change. To be used with Solr search engine since its the only one that handles document churn in a graceful way.

Put errors to the bucket throughout the source code