shoriwe / bftool

A custom worker pool to handle the distribution of function between threads and processes

$ whoami

Hello! I'm Antonio, a Software Engineer, DevOps engineer, and Hacker from Colombia. Here are some highlights about me:

• 🔗 Checkout my Portfolio
• 🚀 I have a strong passion for low-latency solutions.
• 🥷 Information security and horizontal scaling are at the forefront of my concerns.
• 🎆 My personal endeavors mainly revolve around networking, cybersecurity, DeFi and automation.
• 🤝 I'm always keen to contribute to projects that align with my values and beliefs.

🛠️ My tools 🔧

🐱‍💻 Maybe we cross paths at HackTheBox

Stats related to my work

🤔 What kind of projects you will find here?

You will find a proxy, a programming language, a custom function distribution library, python hacks, cool school projects and the list goes on and on. In resume Cool stuff made with passion

Contact

bftool's Issues

`itertools.product` instead of `bftool.CartesianProduct.product`

After some tests and source code review could notice that Python implements it's 'itertools module with C and very well, so the assumptions that it was in pure python are wrong, also it's implementation is 900% faster than mine. Based on that bftool will implement the vanilla itertools.product function now on.

Simpler module interface

By the moment the programmatically way of using bftool is confusing

Arguments Mode printing blank lines

The arguments mode is printing blank lines, maybe a problem when it is checking if the function result is None

Documentation

Create precise documentation of the usage in the Wiki section

1. In code

• Add useful comments in the python code
• Add a rich documentation with python docstring

2. Wiki

• Approach pydoc to generate the necessary files for Wiki

Loading entire file issue

BfTool need a way to handle big files, this because it is hard to load files with more than one thousand words; this feature needs to work like a queue that run in the main program or similar, this way it can be acceded multiple processes and threads at the same time

Pure bruteforce mode

It will be interesting to use a wordlist generated by the main program

Memory argument

Implement an argument that can be of unspecific type to handle the communication between functions

It should be

• Thread safe
• Type independent
• Configurable by argparse

Extreme CPU usage by the completion while-loops

The while loop that wait until active threads are completed consumes a extreme amount of computer resources

`Arguments` as an iter to pass to functions instead of Runner setup

It will be more useful to have the Arguments class as a custom iter instead of the setup for the Runner

Wordlist mode setup by default

Put the wordlist mode as the default mode instead of arguments mode

Using bftool as a script failing

By the moment using:

python3 -m bftool ...

Is not an option. Looks like the last update break this feature, a detailed study is required

Combine wordlist for the same argument

It will be interesting to combine wordlist for the same argument, it means that if the user specify two file wordlist for the same argument, it will combine both, instead of overwriting the first one

Itertools cartesian product resource consumption too high

The python module itertools has a implemented Cartesian product function (itertools.product) that appear to consume a of memory, based on it's implementation

Python official documentation

def product(*args, repeat=1):
    # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
    # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
    pools = [tuple(pool) for pool in args] * repeat
    result = [[]]
    for pool in pools:
        result = [x+[y] for x in result for y in pool]
    for prod in result:
        yield tuple(prod)

It is possible to notice that it first prepare the cartesian product result, and then it iterates over it.

There should be a way to handle the creation of a cartesian product generator with as arguments python generators that don't consume a lot of memory

Currently itertools.product is used in:

• bftool/useful_functions.py:merge_wordlists
• bftool/useful_functions.py:pure_bruteforce_rul
• bftool/WordlistHandler.py:WordlistHandler.setup

No parallel processes are spawned

The -mp flag combined with -m wordlist is not working properly as process are not spawned in parallel

Bruteforce rules from Dict instead of custom string format

It will be more convenient to approach the setup of bruteforce rules by using python's dicts instead of a string with a custom format

Use of asyncio and threading pools to distribute jobs

Distribute jobs approaching asyncio and threading pools instead of the custom one

The implemented queue is not thread safe

The implemented queue to read from the wordlist (generated or file based) is not thread safe, for example two processes can read the same word from it

Example functions to test

Example directory structure:

examples
└── thematic
    └── specific-subject
        ├── README.md
        ├── specific-subject.py
        ├── wordlist-1.txt
        ├── wordlist-2.txt
        └── wordlist-N.txt

The README.md should contain:

• The script Requirements (like external modules)
• The steps to install all the requirements.
• Function Description
• The Command to run the function

Optional

The wordlists are optional as the function can run the generated one