epi052 / feroxbuster

A fast, simple, recursive content discovery tool written in Rust.

A simple, fast, recursive content discovery tool written in Rust

🦀 Releases ✨ Example Usage ✨ Contributing ✨ Documentation 🦀

---

✨🎉👉 NEW DOCUMENTATION SITE 👈🎉✨

🚀 Documentation has moved 🚀

Instead of having a 1300 line README.md (sorry...), feroxbuster's documentation has moved to GitHub Pages. The move to hosting documentation on Pages should make it a LOT easier to find the information you're looking for, whatever that may be. Please check it out for anything you need beyond a quick-start. The new documentation can be found here.

😕 What the heck is a ferox anyway?

Ferox is short for Ferric Oxide. Ferric Oxide, simply put, is rust. The name rustbuster was taken, so I decided on a variation. 🤷

🤔 What's it do tho?

feroxbuster is a tool designed to perform Forced Browsing.

Forced browsing is an attack where the aim is to enumerate and access resources that are not referenced by the web application, but are still accessible by an attacker.

feroxbuster uses brute force combined with a wordlist to search for unlinked content in target directories. These resources may store sensitive information about web applications and operational systems, such as source code, credentials, internal network addressing, etc...

This attack is also known as Predictable Resource Location, File Enumeration, Directory Enumeration, and Resource Enumeration.

⏳ Quick Start

This section will cover the minimum amount of information to get up and running with feroxbuster. Please refer the the documentation, as it's much more comprehensive.

💿 Installation

There are quite a few other installation methods, but these snippets should cover the majority of users.

Kali

If you're using kali, this is the preferred install method. Installing from the repos adds a ferox-config.toml in /etc/feroxbuster/, adds command completion for bash, fish, and zsh, includes a man page entry, and installs feroxbuster itself.

sudo apt update && sudo apt install -y feroxbuster

Linux (32 and 64-bit) & MacOS

Install to a particular directory

curl -sL https://raw.githubusercontent.com/epi052/feroxbuster/main/install-nix.sh | bash -s $HOME/.local/bin

Install to current working directory

curl -sL https://raw.githubusercontent.com/epi052/feroxbuster/main/install-nix.sh | bash

MacOS via Homebrew

brew install feroxbuster

Windows x86_64

Invoke-WebRequest https://github.com/epi052/feroxbuster/releases/latest/download/x86_64-windows-feroxbuster.exe.zip -OutFile feroxbuster.zip
Expand-Archive .\feroxbuster.zip
.\feroxbuster\feroxbuster.exe -V

Windows via Chocolatey

choco install feroxbuster

All others

Please refer the the documentation.

Updating feroxbuster (new in v2.9.1)

./feroxbuster --update

🧰 Example Usage

Here are a few brief examples to get you started. Please note, feroxbuster can do a lot more than what's listed below. As a result, there are many more examples, with demonstration gifs that highlight specific features, in the documentation.

Multiple Values

Options that take multiple values are very flexible. Consider the following ways of specifying extensions:

./feroxbuster -u http://127.1 -x pdf -x js,html -x php txt json,docx

The command above adds .pdf, .js, .html, .php, .txt, .json, and .docx to each url

All of the methods above (multiple flags, space separated, comma separated, etc...) are valid and interchangeable. The same goes for urls, headers, status codes, queries, and size filters.

Include Headers

./feroxbuster -u http://127.1 -H Accept:application/json "Authorization: Bearer {token}"

IPv6, non-recursive scan with INFO-level logging enabled

./feroxbuster -u http://[::1] --no-recursion -vv

Read urls from STDIN; pipe only resulting urls out to another tool

cat targets | ./feroxbuster --stdin --silent -s 200 301 302 --redirects -x js | fff -s 200 -o js-files

Proxy traffic through Burp

./feroxbuster -u http://127.1 --insecure --proxy http://127.0.0.1:8080

Proxy traffic through a SOCKS proxy (including DNS lookups)

./feroxbuster -u http://127.1 --proxy socks5h://127.0.0.1:9050

Pass auth token via query parameter

./feroxbuster -u http://127.1 --query token=0123456789ABCDEF

🚀 Documentation has moved 🚀

For realsies, there used to be over 1300 lines in this README, but it's all been moved to the new documentation site. Go check it out!

✨🎉👉 DOCUMENTATION 👈🎉✨

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Joona Hoikkala 📖	J Savage 🚇 📖	Thomas Gotwig 🚇 📖	Spike 🚇 📖	Evan Richter 💻 📖	AG 🤔 📖	Nicolas Thumann 💻 📖
Tom Matthews 📖	bsysop 📖	Brian Sizemore 💻	Alexandre ZANNI 🚇 📖	Craig 🚇	EONRaider 🚇	wtwver 🚇
Tib3rius 🐛	0xdf 🐛	secure-77 🐛	Sophie Brun 🚇	black-A 🤔	Nicolas Krassas 🤔	N0ur5 🤔 🐛
mchill 🐛	Naman 🐛	Ayoub Elaich 🐛	Henry 🐛	SleepiPanda 🐛	Bad Requests 🐛	Dominik Nakamura 🚇
Muhammad Ahsan 🐛	cortantief 🐛 💻	Daniel Saxton 🤔 💻	n0kovo 🤔 🐛	Justin Steven 🤔	7047payloads 💻	unkn0wnsyst3m 🤔
0x08 🤔	kusok 🤔 💻	godylockz 🤔 💻	Ryan Montgomery 🤔	ippsec 🤔	James 🐛	Jason Haddix 🤔 🐛
Limn0 🐛	0xdf 🐛 🤔	Flangyver 🤔	PeakyBlinder 🤔	Postmodern 🤔	O 💻	John-John Tedro 💻
kmanc 🐛 💻	hakdogpinas 🤔	多可悲 🤔	Aidan Hall 💻 🚇	João Ciocca 🐛 🤔	f3rn0s 🐛	LongCat 🤔
xaeroborg 🤔	Luoooio 🤔	Aan 💻 🚇 🤔	Simon 🐛	Nicolas Christin 🐛	DrDv 🐛	Antoine Roly 🤔
Himadri Bhattacharjee 💻 🤔	Samy Lahfa 🤔	sectroyer 🐛 🤔	ktecv2000 🐛	Andrea De Murtas 💻	sawmj 🐛	Zach Hanson 🐛
Olivier Cervello 🤔	RavySena 🤔	Florian Stuhlmann 🐛	Mister7F 🤔	manugramm 🐛	ArthurMuraro 🐛	Shadow 🐛
dirhamgithub 🐛	FieldOfRice 🚇	Matt 🤔	Sam Leonard 💻	Rewinter 🤔	deadloot 🤔	Spidle 🤔
Julián Gómez 🤔	Petros 🐛

This project follows the all-contributors specification. Contributions of any kind welcome!

feroxbuster's Issues

[FEATURE REQUEST] Pause|resume scan based on keyboard input to terminal

Describe the solution you'd like
I'd like the ability to pause feroxbuster mid-scan via keyboard input and then be able to resume the scan, also via keyboard input

originally suggested by @Flangyver

build rpm/deb packages and put in releases as assets

update installation section in readme

[FEATURE REQUEST] Check for updates on startup

Is your feature request related to a problem? Please describe.
I'd prefer that folks are notified of new releases when they're available.

Describe the solution you'd like
A user runs feroxbuster, if the user's version is behind the current release, notify the user (probably in the banner)

Describe alternatives you've considered
The alternative to always checking would be a --update flag or similar, however, I prefer the auto check

[BUG] Duplicate scans occurring

Describe the bug
In certain situations, duplicate scans are kicking off against the same directory.

Expected behavior
A single scan per directory.

Environment (please complete the following information):

• feroxbuster version: v1.0.1

Additional context
A HashSet of scanned urls would likely solve the problem.

start scan against URL-1
add URL-1 to the url-set
...
new URL found to scan
if new URL in url-set -> do nothing
else -> scan

[FEATURE REQUEST] Response code blacklist, improved white/blacklisting abilities

Ferox should support the ability to blacklist certain response codes, similar to the --hc flag in wfuzz. This would make it drastically easier to hide just one or two codes, versus the current alternative of having to whitelist all the codes you want to see.

fix multi-bar tearing

bars are printed over each other, but shouldn't be

improve integration tests

very little is currently integration tested, any improvement here is beneficial. Will map out a lit of needs later

[FEATURE REQUEST] add replay proxy option

Is your feature request related to a problem? Please describe.
I only want to see unfiltered responses in burp

Describe the solution you'd like
A --replay-proxy option that would only send responses identified as valid (basically not filtered out) to the proxy

Additional context
Thanks to @aringo and @hellor00t for the suggestion

integrate logging with the progress bar printer

logging dorks the progress bar output. the link below can be used as a solution template

https://github.com/getsentry/sentry-cli/blob/master/src/utils/logging.rs

progress bars and wildcard prints need to respect -q

-q should create hidden progress bars

split output streams based on content

banner and bars should print to stderr
url reporting should print to stdout

the indicatif library is making this difficult

[BUG] Excessive CPU usage

When the tool created multiple recursive jobs, at one point, it has output a lot of errors and the CPU usage got to 100% at my machine and it was finally killed.

I have even tried to lower the threads count from 50 to 20 but as the recursive jobs increase I think it didn't really matter

The best way in my opinion to avoid this is to add an argument that takes a number to set the most jobs running at once, and add the new jobs to queue.

Other than that, your tool is awesome, thank you for the efforts!

fix panic caused by calling progress bar::join multiple times

need to add a -v to see it

thread 'tokio-runtime-worker' panicked at 'Already joining!', /home/epi/.cargo/registry/src/github.com-1ecc6299db9ec823/indicatif-0.15.0/src/progress.rs:1035:13

[FEATURE REQUEST] Dynamically tuning concurrency/connection limit or bailing after n connection failures

A common problem I run into is that some sites have issues when supporting too many concurrent connections, or too high a rate of connections- of course every site has its breaking point- but read my next paragraph for more detail on this. This can be mitigated by tuning feroxbuster with -t and -L for each individual target, of course.

The problem comes in when doing testing across a large amount of sites at once, using, e.g. GNU parallel. If you are performing testing against a medium or large organization with many websites, sometimes you'll need to batch a large set of commands due to testing time constraints, and it won't be practical to test and tune the -t and -L setting for each individual site, since they can vary quite a bit within a large set. Consider for this example a list of 1000 or more sites.

A nice feature would be to either:

1. (Simple Solution) Simply bail out after n connection failures
2. (Complex / Better Solution) Tune the threads and/or concurrent connections setting dynamically, based on the occurrence of connection failures

Some workarounds here:

1. As I mentioned, manually testing each site and having a per-site -t and -L setting; this is prohibitively expensive in terms of time during a large-scale test
2. Using a very conservative value across all sites; this is detrimental to the large amount of sites that can handle (in many cases) many multiples of that conservative setting, slowing the entire testing run of the entire batch down significantly

This may be beyond the scope of what you would like to implement and maintain within feroxbuster, but for me, it would be a very useful feature.

Curious what you think about this

Thanks, I appreciate your development on this tool. I haven't seen a public tool that performs as well as feroxbuster, with such flexibility and robust and advanced features since skipfish- which is no longer maintained and never really had a happy medium between "way too agressive" and "completely limited in its findings"

Publish with Homebrew on MacOS & Linux 🍺

I would like to create Brew formulas for MacOS & Linux to make installation easier 🙂

[FEATURE REQUEST] Way to limit scope when using --extract-links

Is your feature request related to a problem? Please describe.
When using --extract-links, it would be nice to have an option which only grabbed links from the original domain. I'm also not sure if it is starting to dir bust on other domains that are extracted? The output is unclear.

Describe the solution you'd like
A flag to limit the scope of the tool would be great. Also additional clarity in the ReadMe on if it starts busting new domains when using the --extract-links option would be great.

P.S. - Absolutely loving the tool! I think you've got a real edge on gobuster & ffuf with this one 👍. I've been sharing will all my colleagues! You've done some really great work on this!

[FEATURE REQUEST] Remove 🎯, 🚀, ...?

Hi, recently, I am testing to see what can feroxbuster do, from my local machine. But, I think that "🎯", "🚀", ... will make feroxbuster like a funny joke script rather than an advanced fuzzer, a complex fuzzing project. Should we delete it?

publish to crates.io

update installation section in README

Remove ansi_term

With the inclusion of console, feroxbuster has two crates that can color the terminal. I use console for other things, so should swap the coloring to use console as well, then remove ansi_term.

Add "no file descriptors available" description and solution to README.md

Depending on a user's operating system's open file limit, the user may see a bunch of no file descriptors available errors. A user can increase the limit by editing limits.conf or using ulimit.

I'd like a description of the problem and both solutions to be included in the readme.

Definition of Done:

• entry in table of contents, something like Common Problems or similar
• sub-entry in table of contents to the above that indicates this particular problem (no file descriptors available)
• H2 heading that corresponds to the Common Problems or similar entry (include relevant emoji, warning symbol or something?)
• H3 heading that corresponds to the particular problem
• Under the H3 heading, a description of the problem and both limits.conf and ulimit solutions

Relevant discussions:

• #77 (comment)
• https://forum.hackthebox.eu/discussion/comment/85185

remove blocking wordlist read

stream the wordlist to scanning as i read it in, instead of blocking

[FEATURE REQUEST] Cancel threads from a recursive search.

Is your feature request related to a problem? Please describe.
Nope; actually I came across this last week and love it.

Describe the solution you'd like
The ability to cut a thread on a scan. So for instance, if I find a hidden CMS solution on a port at /wordpress/ and the scan begins scanning the /wordpress/ directory to stop the scan of the original directory. I get I could cancel or start a new instance but with application searching, you could filter your recursive searching to cut down on time. I guess you could also pipe it into an additional scan?

Describe alternatives you've considered
Cancelling the current scan and starting a new one.

Additional context
None.

[BUG] SHA256 mismatch for MacOS

Describe the bug
Using the 2 provided commands to install on Mac gives a SHA256 mismatch

To Reproduce
Steps to reproduce the behavior:

1. brew tap tgotwig/feroxbuster
2. brew install feroxbuster

Expected behavior
A successful install

Traceback / Error Output
brew install feroxbuster
Updating Homebrew...
==> Installing feroxbuster from tgotwig/feroxbuster
==> Downloading https://raw.githubusercontent.com/epi052/feroxbuster/master/ferox-config.toml.example
######################################################################## 100.0%
Error: SHA256 mismatch
Expected: 70ace4e70c7f532cc4f7e7958106d035c62bd9d12a6a91de433b815f607911ba
Actual: d53171328e75472612470e337ec374376ede56631d12129aecb1cd29fefb69b8

Environment (please complete the following information):

• feroxbuster version: [ v1.4.0]
• OS [mac OS Catalina 10.15.7]

Add strip to CD pipeline

For both linux binaries, strip -s should be run on the binary before upload in order to reduce final binary size.

If there is a similar command for macos, include that in the macos build.

improve unit tests

write unit tests for create_urls

Timeout Issue

Is your feature request related to a problem? Please describe.

When you fuzz you need the most confortable output in order to analyze the results
When the request timeout it's shows in the output without any -v used

Describe the solution you'd like
I would like if possible a flag to supress this timeout urls and leave only the status code and useful information in the output

Screenshot

Fuzzing terminate due to unknow stack trace error

Describe the bug
Hello
Thank you for the awesome tool

I've been encountering this issue when fuzzing a url it abort the fuzzing and throw so unknow errors
To Reproduce
Steps to reproduce the behavior:

1. Downloaded the latest relases
2. Made an alias for automation looks like this

fastfuzz> # Defined in /root/.config/fish/functions/fastfuzz.fish @ line 1
          function fastfuzz
              feroxbuster --url $argv -w /opt/SecLists/Discovery/Web-Content/dic.txt -d 3 -x html,aspx,php,asp,log
          end

Works in fish
3. I run fastfuzz

Expected behavior
Bruteforce recursively and print out results

Traceback / Error Output

[>-------------------] - 8m     44744/1018686 85/s    http://95.163.33.203/
thread 'tokio-runtime-worker' panicked at 'Already joining!', /github/home/.cargo/registry/src/github.com-1ecc6299db9ec823/indicatif-0.15.0/src/progress.rs:1035:13
stack backtrace:
   0:           0x679af8 - <unknown>
   1:           0x48d8ec - <unknown>
   2:           0x6791a6 - <unknown>
   3:           0x678baa - <unknown>
   4:           0x678431 - <unknown>
   5:           0x533e94 - <unknown>
   6:           0x53fbca - <unknown>
   7:           0x4e8dea - <unknown>
   8:           0x68d290 - <unknown>
   9:           0x68d025 - <unknown>
  10:           0x6873c5 - <unknown>
fish: Job 1, 'feroxbuster --url $argv -w /opt…' terminated by signal SIGABRT (Abort)

Environment (please complete the following information):

• feroxbuster version: 1.0.0
• OS [WSL 1]

[FEATURE REQUEST] Word count / line count filters - LMAY75

The ability to white/blacklist based off of characters/line count on the response page would be greatly appreciated. Sometimes you won't find what you want from a response code, but will from a char or line count. Both features are in wfuzz for comparison.

Originally reported by @LMAY75

Links don't show up at the top like in the example[BUG]

When I run FeroxBuster with default settings, my output is full of status bars instead of all of the directories listed at the bottom. Is there a setting that I'm missing? I'm on Kali 2020, with the latest release 1.2.0

See Below:

Thank you

[ENHANCEMENT] Homebrew installations should place a config file during installation

Hi

I just installed the tool via Homebrew on Mac

Now I need help to find the ferox-config.toml on my Mac, to add the wordlist path.

Thanks

[FEATURE REQUEST] Create Snap package

Is your feature request related to a problem? Please describe.
I'd like to offer snap installation as an option for feroxbuster.

Describe the solution you'd like
sudo snap install feroxbuster

If anyone decides to work on this other than me, a snapcraft.yaml would satisfy this issue. From there I can handle publishing etc...

https://snapcraft.io/docs/rust-applications

[BUG] Progress bar not incremented correctly when using -x

Describe the bug
When -x is used, the scanner only increments the counter when a 'base' request is made. It exits shortly after the number of requests reach roughly the length of the wordlist. This indicates that requests generated from extensions aren't incrementing the progress bar as expected.

To Reproduce
Steps to reproduce the behavior:

1. feroxbuster -u SOMEURL -x php

Take note of the # of words in the wordlist and the # of requests sent. Expect exit shortly after requests reach # of words.

Expected behavior
All requests should increment progress the bar, not only the base requests.

Environment (please complete the following information):

• feroxbuster version: 1.0.0 - 1.1.1
• OS:
  • kali 2020.2
  • ubuntu 20.04

[BUG] Properly notify users of bad certs

Describe the bug
When scanning a site that doesn't have a valid cert, the error message doesn't let the user know that the problem is certs. Instead, all that's shown is that ferox can't connect

To Reproduce
Steps to reproduce the behavior:

1. scan a site with an invalid cert
2. see ERROR heuristics::connectivity_test Could not connect to any target provided

Expected behavior
I expect a clear message notifying the user that the certificate is invalid

Environment (please complete the following information):

• feroxbuster version: v1.4.0

Additional context
Thanks to @Decap1tator for pointing out the issue

formalize output

• put url at the end of the line instead of the middle
• give size a normalized size with string formatting
• decide whether to keep canonical status code reason in output or shrink to just the 2xx/3xx/4xx etc codes

[BUG] Erroneous output to terminal when using --extract-links

Is your feature request related to a problem? Please describe.
When using --extract-links, it would be nice to have an option which only grabbed links from the original domain. I'm also not sure if it is starting to dir bust on other domains that are extracted? The output is unclear.

Describe the solution you'd like
A flag to limit the scope of the tool would be great. Also additional clarity in the ReadMe on if it starts busting new domains when using the --extract-links option would be great.

P.S. - Absolutely loving the tool! I think you've got a real edge on gobuster & ffuf with this one 👍. I've been sharing will all my colleagues! You've done some really great work on this!

[ENHANCEMENT] Increase test coverage

A blanket issue for test coverage improvements

[FEATURE REQUEST] -q Flag (same detail, less frequent updates)

Is your feature request related to a problem? Please describe.
Nope. Refer to previous post on the love of this script.

Describe the solution you'd like
To change (or have a different -q flag) for:

Describe alternatives you've considered
Using -q works but I would prefer to have all the information but the time visible. Or to show progress only once it has found a URL or String.

Additional context
My reasoning for this is I use TMUX and I have it set to notify me on panes when something appears in the HUD. Useful for notifying me well-running an HTTP server and picking up the request... or doing further enumeration well exploring other ports. Having this feature would mean I would only get a notification when a URL/string was found.

add initialize function to config

this is more of a 'can we' issue

can the lazy_static call be housed inside an initialize function for conformity?

[ENHANCEMENT] CI/CD should create a x86_64 linux tarball

https://github.com/epi052/feroxbuster/blob/master/.github/workflows/build.yml needs to be updated such that the x86_64 build creates a tar.gz file as well. This will be used for homebrew installs.

An example already exists in the build-macos job. https://github.com/epi052/feroxbuster/blob/master/.github/workflows/build.yml#L62

This issue is the mirror of that for 64 bit linux builds.

[FEATURE REQUEST] Add LinkFinder-esque capability to find additional content

Describe the solution you'd like
As valid 2xx responses are found, examine their contents for additional files/directories.

All directories found should be added to new recursive scans, as long as they don't exceed the recursion depth limit. If a newly discovered directory is found and exceeds the limit, the user should still be notified.

Extracting links should be an opt-in feature, as there's a cost for the additional coverage provided.

Additional context
The feature branch for this is tracked in the linked pull request

[FEATURE REQUEST] Check for ferox-config.toml in multiple locations

Describe the solution you'd like
feroxbuster should look for its config in the same directory as the tool (current), ~/.config/feroxbuster, and cwd

Make output file write async

writing to output file is not async, make it so.

in doing so, i'd prefer the output file to be sorted by url before the program exits via a cleanup section (indicating to the user that scans are done and we're performing clean up things)

[FEATURE REQUEST] Create package suitable for PPA

Is your feature request related to a problem? Please describe.
I'd like to distribute feroxbuster through a PPA. Long-term, i'd love to see it integrated into an official distro's repo (debian / ubuntu / kali).

Describe the solution you'd like

sudo add-apt-repository ppa:SOME_PPA
sudo apt update
sudo apt install feroxbuster

[FEATURE REQUEST] Allow -v induced logging to be captured in a file

Is your feature request related to a problem? Please describe.
Due to how MultiProgress handles printing when not in a user_attended shell, there is no simple way to capture the output generated from -v+.

Describe the solution you'd like
A way for those logs to be captured in a file. I think a --logfile makes sense, but am open to suggestions.

add badges to readme

Limit number of recursive scans with a Queue

Is your feature request related to a problem? Please describe.
Per this issue, excessive CPU usage is a possibility given the current implementation of recursion.

Describe the solution you'd like
Limit the number of recursive scans by using a consumer/producer w/ queue system. The limit should have a sane default and be configurable from the command line/config file.

readme needs updated

• missing config options
• better ToC
• comparison matrix
• etc...

[FEATURE REQUEST] Automatically adjust nofile limit on linux/macos

Is your feature request related to a problem? Please describe.
As discussed here and here, low limits on number of open files allowed by the OS can result in spurious errors reported to the user.

Describe the solution you'd like
Use setrlimit syscall to adjust the number of open files allowed, if necessary.

• The limit should be reset to normal once complete
• will likely need to inspect the hard limit with getrlimit in order to know the max adjustment allowed for an unprivileged user
• consider notifying the user of the change

fix musl CI/CD build

musl build on github's pipeline fails due to openssl dependency

Add Option Groups to parser

I'd like to have groups of options visually ... grouped together. Maybe not exactly these groups, but something along these lines. Currently clap doesn't offer this functionality but will in version 3.0.

Once clap 3.0 is released, this ticket can be completed.

Filter Options:
    -S, --filter-size <SIZE>...             Filter out messages of a particular size (ex: -S 5120 -S 4927,1970)
    -C, --filter-status <STATUS_CODE>...    Filter out status codes (deny list) (ex: -C 200 -C 401)
...

 Include Options:
    -s, --status-codes <STATUS_CODE>...     Status Codes to include (allow list) (default: 200 204 301 302 307 308 401)
...

Scan Options:
...

Client Options:
...