flipkart-incubator / rta

Red team Arsenal - An intelligent scanner to detect security vulnerabilities in company's layer 7 assets.

Red Team Arsenal 

Red Team Arsenal is a inhouse framework created by flipkart security team which monitors the external attack surface of the company's online assets and provide an holistic security view of any security anomalies. It's a closely linked collection of various security engines and tools to conduct/simulate attacks and monitor public facing assets for anomalies and leaks.

It's an intelligent framework that detects security anomalies in all layer 7 assets and reports the same.

As companies continue to expand their footprint on INTERNET via various acquisitions and geographical expansions, human driven security engineering is not scalable, hence, companies need feedback driven automated systems to stay put.

Installation

Supported Platforms

RTA has been tested both on Ubuntu/Debian (apt-get based distros) distros, it will also work with Mac-OS (but do replace the binaries folder with the binaries of the programs that match with your system architecture if you are running it on Mac OS)

Prerequisites:

There are a few packages which are necessary before proceeding with the installation:

• Python 3.8, which is installed by default in most systems
• Python pip: sudo apt-get install python3-pip

Also, you will need to update the config file and config_mail yml files with the following data

• Google Chat Webhook URL
• SMTP Credentials
• Github Token
• Mailing List
• Custom Search Engine (API Key)

You can install all the requirements by running sudo pip3 install -r requirements.txt

Usage

Mention the domains you want to monitor inside 'input_files/domains.txt'

python3 scheduler.py

Configuring Google Chat:

You can receive notifications in Google Chat, this can be configured by adding the webhook URL to config.yml file.

Contributors

• Anirudh Anand (@a0xnirudh)
• Mohan KK (@MohanKallepalli)
• Ankur Bhargava (@_AnkurB)
• Prajal Kulkarni (@prajalkulkarni)
• Himanshu Kumar Das (@mehimansu)
• Mandeep Jadon (@1337tr0lls)
• Vivek Srivastav (@vivek_15887)
• Abhishek S (@abhiabhi2306)

References

Thanks to the authors of the below creations.

• Amass: https://github.com/owasp-amass/amass
• Aquatone: https://github.com/michenriksen/aquatone
• Assetfinder: https://github.com/tomnomnom/assetfinder
• Chaos-Client: https://github.com/projectdiscovery/chaos-client
• Chrome Driver: https://chromedriver.chromium.org/downloads
• Ffuf: https://github.com/ffuf/ffuf
• Finddomain: https://github.com/Findomain/Findomain
• Gau: https://github.com/lc/gau
• Https: https://github.com/projectdiscovery/httpx
• Naabu: https://github.com/projectdiscovery/naabu
• Nmap: https://nmap.org/
• Subfinder: https://github.com/projectdiscovery/subfinder
• Scweet: https://pypi.org/project/Scweet/
• Cloud Enum: https://github.com/initstring/cloud_enum
• DNStwist: https://github.com/elceef/dnstwist
• Phonebook: https://phonebook.cz/
• SSLyze: https://github.com/nabla-c0d3/sslyze
• Shodan: https://shodan.io

rta's Issues

nessus scan status err

Traceback (most recent call last):
File "rta.py", line 416, in
main()
File "rta.py", line 410, in main
scan.nessus_scan(recon.subdomains, filename)
File "rta.py", line 304, in nessus_scan
self.nessus.scan_results(filename)
File "/root/RTA/scanning/nessus.py", line 197, in scan_results
if response["status"] == "ready":
KeyError: 'status'

Python version:
Python 2.7.12

OS, kernel:
Linux rta 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Scan stopped with bellow error

Traceback (most recent call last):
File "rta.py", line 416, in
main()
File "rta.py", line 403, in main
scan.wp_scan(args.url)
File "rta.py", line 361, in wp_scan
dataid = collection.insert(data)
UnboundLocalError: local variable 'data' referenced before assignment

localhost:27017: [Errno 111] Connection refused

hi getting this error:

$sudo python rta.py --url "vulnweb.com" -v -s

fTraceback (most recent call last):
File "rta.py", line 416, in
main()
File "rta.py", line 389, in main
recon = Recon()
File "rta.py", line 55, in init
self.scraper = scraper.Scraper()
File "/home/apps/gits/RTA/recon/scraper/scraper.py", line 37, in init
self.collection.create_index([("url", DESCENDING), ("hostname", ASCENDING)], unique=True)
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 1387, in create_index
self.__create_index(keys, kwargs)
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 1294, in __create_index
with self._socket_for_writes() as sock_info:
File "/usr/lib/python2.7/contextlib.py", line 17, in enter
return self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 762, in _get_socket
server = self._get_topology().select_server(selector)
File "/usr/local/lib/python2.7/dist-packages/pymongo/topology.py", line 210, in select_server
address))
File "/usr/local/lib/python2.7/dist-packages/pymongo/topology.py", line 186, in select_servers
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused

Kali 2018 issue

Hello i am having a problem like this:

Traceback (most recent call last):
File "rta.py", line 20, in
from recon.scraper import scraper
File "/root/RTA/recon/scraper/scraper.py", line 7, in
from twitter import OAuth
ImportError: cannot import name OAuth

How to run without leaving old subdomains out

What's the easiest way to run a new subdomain report each time, irrespective of the last run? I want to be able to see the current results in Slack each time it is run. This is awesome btw. I wrote a similar one using Aquatone, and Qualys and AWS APIs. Didn't have Slack integration and that's how I got here. Thanks!

scan.wp_scan(args.url) bug

Traceback (most recent call last):
File "rta.py", line 417, in
main()
File "rta.py", line 404, in main
scan.wp_scan(args.url)
File "rta.py", line 330, in wp_scan
data['version'] = result['version']['number']
TypeError: 'NoneType' object has no attribute 'getitem'

Python version:
Python 2.7.12

OS, kernel:
Linux rta 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Fix:
replace line 404 in rta.py
scan.wp_scan(args.url)
with
scan.wp_scan("--url " + args.url)