whiteeyehansel / maltelligence

a Malware/Threat Analyst Desktop

#Maltelligence ###a Malware/Threat Analyst Desktop

##About

Maltelligence is a tool developed by Maltelligence Research Group to automatically collect malicious network infrastructure information and malware samples RECURSIVELY from various open source intelligence (OSINT) sources including virustotal, whois, passive DNS, IP subnets, AS number and Geolocation information.

You may run Maltelligence PERIODICALLY to capture and profile the behaviour of malicious group of domain, IP, whois and html content along different stage/time of APT attacks.

Maltelligence is a project from the insight of MalProfile

##Installation If you want to run your instance of Maltelligence locally on your machine, be sure you have the following requirements installed:

###Requirements

Below procedure is tested in Ubuntu 14.04

• Mysql installation

  The root password configured will be used in MalProfile.ini

  sudo apt-get install mysql-client-core-5.6
  sudo apt-get install mysql-server-5.6

• Install dependencies

   sudo apt-get install git
   sudo apt-get install python-setuptools
   sudo apt-get install build-essential python-dev libmysqlclient-dev
   sudo apt-get install libfuzzy-dev
   sudo easy_install pip

   mkdir download
   cd download

   wget http://sourceforge.net/projects/ssdeep/files/ssdeep-2.13/ssdeep-2.13.tar.gz/download 
   tar xvfz download
   cd ssdeep-2.13/
   ./configure
   make
   make check
   sudo make install
   
   cd..
   git clone https://github.com/kbandla/pydeep.git
   cd pydeep
   python setup.py build
   sudo python setup.py install

• Install python modules

   sudo pip install MySQL-python
   sudo pip install ipaddress
   sudo pip install pyprind
   sudo pip install mechanize
   sudo pip install dnspython
   sudo pip install pythonwhois
   sudo pip install ipwhois
   sudo pip install beautifulsoup4
   sudo pip install simplejson
   sudo pip install prettytable
   sudo pip install geoip2
   sudo pip install wget

###Installation

Assume you use ~/Malyzer/maltelligence as the program folder

   mkdir Malyzer
   cd Malyzer
   git clone git://github.com/maltelligence/maltelligence.git
   cd maltelligence
   mkdir log
   mkdir repo
   cd..

###Configuration

• Prepare Mysql database

  ONLY drop the "maltelligence"" database if it exists

   mysql -u root -p -e "drop database maltelligence"

Create the schema and import the database

   mysql -u root -p -e "create schema maltelligence default character set utf8"
   mysql -u root -p maltelligence < ./db/maltelligence.sql

• MalProfile.ini

  Obtain a public API key from VirusTotal and put it on (THIS IS COMPULSORY!!!)(NO single or double quotes are required!!!)

   VT_APIKEY=

If you have API key in passivedns.mnemonic.no for passivedns query,

   MN_APIKEY=

VTLIMIT = True means Maltelligence will stop query when VTDEPTH is reached

   VTLIMIT=True

If one domain query from virustotal returns 4 IP addresses, VTDEPTH = 1(domain)+4(IP) = 5 (If VTDEPTH is too large 100 or more, the query time may be extremely long and the APIKEY may be blocked!)

   VTDEPTH=15

If you have subcripitons on tcpiputils.com Premium account for AS Number query, use firefox to signin to the website with your acocunt, then copy the cookie.sqlite file from firefox under Maltelligence folder then setup the below parameter.

   ASN=True

If you want to add Geolocation based on IP addresses

   GEOIP=True

If you want to display the Maltelligence logo

   LOGO=True

If you want to disable collection of nonroutable IP addresses (e.g. 127.0.0.1, 192.168.0.0/24)

   EXCLUDE_NONROUTABLE=True

If you want to add additional Top Level Domain

   TLD=

Fill in the Mysql database information under [MALTELLIGENCE] session

   DB_HOST=localhost
   DB_ID=root
   DB_PW=password
   DB=maltelligence

• Sample MalProfile.ini config:

---------------------------------------------------------------------------------   
[API_KEYS]
VT_APIKEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
MN_APITKEY=

[VT_4]
VTLIMIT=True
VTDEPTH=15

[MALTELLIGENCE]
ASN=False
GEOIP=True
LOGO=True
EXCLUDE_NONROUTABLE=True
DB_HOST=localhost
DB_ID=root
DB_PW=password
DB=maltelligence
TLD=AC,ACADEMY,ACTOR,AD,AE,AERO,AF,AG,AGENCY,AI,AL,AM,AN,AO,AQ,AR,ARPA,AS,ASIA,...
----------------------------------------------------------------------------------

• Geolocation database setup

  Download (and unzip) Geolocation data from MaxMind here: http://dev.maxmind.com/geoip/geoip2/geolite2/ http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz Put it under the Maltelligence folder ~/Malyzer/Maltelligence/ as

   ~/Malyzer/maltelligence/GeoLite2-City.mmdb

• Setup the shell to include Maltelligence path at the bottom of .profile file

   vi ~/.profile
   export PATH=$PATH:$HOME/Malyzer/maltelligence

• Database clean up

  Maltelligence comes with a sample case from Xecure incident, if you want to remove this case or start from scratch

   ./utils/clean_up.sh

Usage

Maltelligence composes of two main python script: Maltelligence.py and report.py

Maltelligence.py - script for OSINT collection report.py - script for basic reporting

Maltelligence group all data in a case as tag For each case, you need to provide at least one sample, one domain and one IP addresses If you don't have a sample, you could still collect the domains and IP addresses information

#Maltelligence.py

• To get help

   python Maltelligence.py -h
   python report.py -h

To obtain the syntax of specific function, just run the related command:

   python Maltelligence.py -d

• Batch collection based on groups of domain, IP and hashes in file

  For format of data file, please refer to readme/*.txt, any combination of IP,domain and hashes can be used

  python Maltelligence.py -m -b --path ./readme/Xecure.txt --tag Xecure

• To update the database with a malicious FQDN/domain/ip (e.g. www.maltelligence.org)

   python Maltelligence.py -u --target www.maltelligence.org --tag SampleCase
   python Maltelligence.py -c --target x.x.x.x --tag SampleCase 

• To create/update the database with a malicious FQDN/domain/ip AND malware samples (e.g. www.maltelligence.org)

   python Maltelligence.py -c --tag SampleCase --target x.x.x.x www.maltelligence.org --hash  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

• To download a malware sample

   python Maltelligence.py -d --hash  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --tag SampleCase

• To recursively collecting the domain/ip information

   python Maltelligence.py -q --target www.maltelligence.org 

• To update Parked domain from IP (IP can be a subet with format 192.168.0.0/24)

   python Maltelligence.py -p ip --target x.x.x.x/y 

• To download the web page of malicious Domain

   python Maltelligence.py -w --target www.maltelligence.org

#report.py

• To show summary of current collected data

   python report.py -t

• To show case details in database

   python report.py -c

• To show date and IP details of specific domain

   python report.py -d --target www.maltelligence.org

• To show date and domain details of specific IP

   python report.py -i --target x.x.x.x

• To show passive DNS report of specific domain

   python report.py -p --target www.maltelligence.org

• To show whois report of specific domain

   python report.py -w --target www.maltelligence.org

• To show domain of same register (search contain input string)

   python report.py -r registername (search contain similar string)

• To show domain of same email (search contain similar string)

   python report.py -e emailaddress

• To show domain of same case (search contain similar string)

   python report.py -s casename

• Command like -d and -i can include --hash to get the malware related to specific domain/IP

   python report.py -d --target www.maltelligence.org --hash

##Roadmap

• New core engine to improve data collection
• Elasticsearch/Logstash/Kibana (ELK) migration

##Team

Maltelligence has been developed and maintained at Maltelligence Research Group by:

Frankie Michael Dan Leng Frank

If you want to know more about Maltelligence, how it works and future developments, please visit the Blog. For any specific request or comment we suggest you to use Github. If none of these worked for you, you can write us at [email protected].

##License

Maltelligence is provided under the GPL (General Public License) v.3:

Copyright (c), 2015 Maltelligence Research Group

<[email protected]>  
 
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.