GithubHelp home page GithubHelp logo

xnl-h4ck3r / xnlinkfinder Goto Github PK

View Code? Open in Web Editor NEW
1.1K 1.1K 143.0 12.36 MB

A python tool used to discover endpoints, potential parameters, and a target specific wordlist for a given target

Python 100.00%

xnlinkfinder's Introduction

If my tools help you find any great bounties, consider BUYING ME A COFFEE ☕ (I could use the caffeine!)...

Twitter  Mastodon

 

Buy Me a Coffee at ko-fi.com

xnlinkfinder's People

Contributors

fabaff avatar xnl-h4ck3r avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

xnlinkfinder's Issues

cannot install

┌──(kali㉿kali)-[~/xnLinkFinder]
└─$ sudo python setup.py install
running install
/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

    ********************************************************************************
    Please avoid running ``setup.py`` directly.
    Instead, use pypa/build, pypa/installer, pypa/build or
    other standards-based tools.

    See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
    ********************************************************************************

!!
self.initialize_options()
/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

    ********************************************************************************
    Please avoid running ``setup.py`` and ``easy_install``.
    Instead, use pypa/build, pypa/installer, pypa/build or
    other standards-based tools.

    See https://github.com/pypa/setuptools/issues/917 for details.
    ********************************************************************************

!!
self.initialize_options()
running bdist_egg
running egg_info
writing xnLinkFinder.egg-info/PKG-INFO
writing dependency_links to xnLinkFinder.egg-info/dependency_links.txt
writing requirements to xnLinkFinder.egg-info/requires.txt
writing top-level names to xnLinkFinder.egg-info/top_level.txt
reading manifest file 'xnLinkFinder.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'xnLinkFinder.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/xnLinkFinder
copying build/lib/xnLinkFinder/init.py -> build/bdist.linux-x86_64/egg/xnLinkFinder
copying build/lib/xnLinkFinder.py -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/xnLinkFinder/init.py to init.cpython-311.pyc
byte-compiling build/bdist.linux-x86_64/egg/xnLinkFinder.py to xnLinkFinder.cpython-311.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
pycache.xnLinkFinder.cpython-311: module references file
creating 'dist/xnLinkFinder-4.1-py3.11.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing xnLinkFinder-4.1-py3.11.egg
removing '/usr/local/lib/python3.11/dist-packages/xnLinkFinder-4.1-py3.11.egg' (and everything under it)
creating /usr/local/lib/python3.11/dist-packages/xnLinkFinder-4.1-py3.11.egg
Extracting xnLinkFinder-4.1-py3.11.egg to /usr/local/lib/python3.11/dist-packages
Adding xnLinkFinder 4.1 to easy-install.pth file

Installed /usr/local/lib/python3.11/dist-packages/xnLinkFinder-4.1-py3.11.egg
Processing dependencies for xnLinkFinder==4.1
Searching for urlparse3
Reading https://pypi.org/simple/urlparse3/
Downloading https://files.pythonhosted.org/packages/10/0e/7645258aada05f6145ff60ee4d117d06400afea1039d3d15a18b71a06176/urlparse3-1.1.tar.gz#sha256=0e2989f3ba54723d9c6607e72a71100b9b6d6efaacb0800480e6b7ef5f5ff5d0
Best match: urlparse3 1.1
Processing urlparse3-1.1.tar.gz
Writing /tmp/easy_install-y1dk70wk/urlparse3-1.1/setup.cfg
Running urlparse3-1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-y1dk70wk/urlparse3-1.1/egg-dist-tmp-1bpjslw2
/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

    ********************************************************************************
    Please avoid running ``setup.py`` directly.
    Instead, use pypa/build, pypa/installer, pypa/build or
    other standards-based tools.

    See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
    ********************************************************************************

!!
self.initialize_options()
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 156, in save_modules
yield saved
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 198, in setup_context
yield
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 259, in run_setup
_execfile(setup_script, ns)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-y1dk70wk/urlparse3-1.1/setup.py", line 3, in

File "/usr/lib/python3/dist-packages/setuptools/init.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
self.finalize_options()
File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 17, in finalize_options
self.set_undefined_options('install',('install_layout','install_layout'))
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
setattr(self, dst_option, getattr(src_cmd_obj, src_option))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in getattr
raise AttributeError(attr)
AttributeError: install_layout. Did you mean: 'install_platlib'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/kali/xnLinkFinder/setup.py", line 4, in
setup(
File "/usr/lib/python3/dist-packages/setuptools/init.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 80, in run
self.do_egg_install()
File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 137, in do_egg_install
cmd.run(show_deprecation=False)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 440, in run
self.easy_install(spec, not self.no_deps)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 682, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 729, in install_item
self.process_distribution(spec, dist, deps)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 774, in process_distribution
distros = WorkingSet([]).resolve(
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 827, in resolve
dist = self._resolve_dist(
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 863, in _resolve_dist
dist = best[req.key] = env.best_match(
^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1133, in best_match
return self.obtain(req, installer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 1145, in obtain
return installer(requirement)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 701, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 727, in install_item
dists = self.install_eggs(spec, download, tmpdir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 920, in install_eggs
return self.build_and_install(setup_script, setup_base)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1194, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1178, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 249, in run_setup
with setup_context(setup_dir):
File "/usr/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 190, in setup_context
with save_modules():
File "/usr/lib/python3.11/contextlib.py", line 155, in exit
self.gen.throw(typ, value, traceback)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 169, in save_modules
saved_exc.resume()
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 143, in resume
raise exc.with_traceback(self._tb)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 156, in save_modules
yield saved
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 198, in setup_context
yield
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 259, in run_setup
_execfile(setup_script, ns)
File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 46, in _execfile
exec(code, globals, locals)
File "/tmp/easy_install-y1dk70wk/urlparse3-1.1/setup.py", line 3, in

File "/usr/lib/python3/dist-packages/setuptools/init.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 164, in run
cmd = self.call_command('install_lib', warn_dir=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/command/bdist_egg.py", line 150, in call_command
self.run_command(cmdname)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
self.finalize_options()
File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 17, in finalize_options
self.set_undefined_options('install',('install_layout','install_layout'))
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
setattr(self, dst_option, getattr(src_cmd_obj, src_option))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in getattr
raise AttributeError(attr)
AttributeError: install_layout

let xnLinkFinder parse folder recursively :)

I tried run xnLinkFinder on jadx output folder but I got
There are no files with a size greater than 500 Mb to process
so any chance to run xnLinkFinder without tracking any size e.g.


for directory,directoriesnames,filesnames in os.walk(args.directory):
    for filename in filesnames:
        filelookup = os.path.join(directory,filename)
        if os.path.isfile(filelookup):
            with open(filelookup,encoding='utf8',errors='ignore') as lines:
                for line in lines:
                    match = regex.search(line)
                    if match:
                        print (match.string[match.start():match.end()])

Couldn't find index page for 'urllib' (maybe misspelled?) while trying to install in python 3.9.1

Hi @xnl-h4ck3r,

I tried to do a fresh install in intel mac pro using python 3.9.1 but i got the following error

Searching for urllib
Reading https://pypi.org/simple/urllib/
Couldn't find index page for 'urllib' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
No local packages or working download links found for urllib
error: Could not find suitable distribution for Requirement.parse('urllib')

I was able to install it successfully after changing urllib here

install_requires=["argparse","requests","psutil","pyyaml","termcolor","urlparse3","beautifulsoup4","lxml","html5lib","urllib"],
to urllib3.

Support optional creation of separate, wordlist-like output for found links

Thanks for putting out this great tool @xnl-h4ck3r.

I think it'd be useful to implement an optional feature which does a (small) amount of post-processing on found links, then writing this output to a separate, wordlist-like file in conventional format (e.g. line-separated example.html, api/v1/users etc). This would allow for easier chaining with other tools (e.g. passing this file to ffuf or Intruder) or creating/appending to custom wordlists using this output.

I'd be happy to submit a pull request which adds this feature if you think it'd be worthwhile (I implemented something similar on a private rework of GerbenJavado's LinkFinder and found it to be handy)!

REGEX issue!

Any way to grep only parameters, like ?url=. My regex is (\?\w+=)

image

Flag to auto-accept or auto-decline 'Do you want to remove tags form the file?'

When analyzing burp data files the xnLinkFinder asks the user for

Do you want to remove tags form the file?

There is no way to accept this option in script so xnLinkFinder cannot be used in scripts.

For example I'm using xnLinkFinder (and other tools) in docker containers. I'm using gnu parallels to spin up multiple docker containers with xnLinkFinder each analyzing one burp data file. This is currently not possible because if you run docker within gnu parallels you can't use '-i -t' (interactive + tty) flags.

Question

I possible version for owasp zap?

Having problems installing on Ubuntu 22.04 lts

I have fresh install of ubuntu 22.04 lts
When i run the setup.py script, I receive the following output and error

pwn4g3@Machine:~/tools/xnLinkFinder$ sudo python3 setup.py install
running install
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
creating xnLinkFinder.egg-info
writing xnLinkFinder.egg-info/PKG-INFO
writing dependency_links to xnLinkFinder.egg-info/dependency_links.txt
writing requirements to xnLinkFinder.egg-info/requires.txt
writing top-level names to xnLinkFinder.egg-info/top_level.txt
writing manifest file 'xnLinkFinder.egg-info/SOURCES.txt'
reading manifest file 'xnLinkFinder.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'xnLinkFinder.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
copying xnLinkFinder.py -> build/lib
creating build/lib/xnLinkFinder
copying xnLinkFinder/__init__.py -> build/lib/xnLinkFinder
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/xnLinkFinder
copying build/lib/xnLinkFinder/__init__.py -> build/bdist.linux-x86_64/egg/xnLinkFinder
copying build/lib/xnLinkFinder.py -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/xnLinkFinder/__init__.py to __init__.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/xnLinkFinder.py to xnLinkFinder.cpython-310.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying xnLinkFinder.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
__pycache__.xnLinkFinder.cpython-310: module references __file__
creating dist
creating 'dist/xnLinkFinder-4.4-py3.10.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing xnLinkFinder-4.4-py3.10.egg
creating /usr/local/lib/python3.10/dist-packages/xnLinkFinder-4.4-py3.10.egg
Extracting xnLinkFinder-4.4-py3.10.egg to /usr/local/lib/python3.10/dist-packages
Adding xnLinkFinder 4.4 to easy-install.pth file

Installed /usr/local/lib/python3.10/dist-packages/xnLinkFinder-4.4-py3.10.egg
Processing dependencies for xnLinkFinder==4.4
Searching for lxml
Reading https://pypi.org/simple/lxml/
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning:  is an invalid version and will not be supported in a future release
  warnings.warn(
Downloading https://files.pythonhosted.org/packages/83/61/88abc612c710512fe6de106b33cffb1c835689a590f65ce5e6ad2401ffca/lxml-5.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl#sha256=b4eef43c5dc5c579d0804e55a32dd1bacbd008c8191ed4d65be278bbb11ddc61
error: The read operation timed out

thanks

Rate limit and custom User-Agent header

Hi @xnl-h4ck3r,

Thanks for your great work. It would be useful to add the following two options to the tool, if possible:

  • rate limit option specifying the maximum requests to send per second (in addition to the existing -p option)
  • custom User-Agent header (it doesn't seem to be possible using the existing -u or -H options)

Kind regards,
abitsec

Problem not crawling some relative paths

hello, I'm using your tools well.

but there are problem not crawling some relative paths.

The problem occurs when a relative path has not prefix.
ex) path/example/test

I'm using it after fixing it like this, is there any better way?
image

Tool throws a "killed" message

Hello,
I'm running xnlinkfinder on the waymore directory for a target and while doing the same, the utility processes the responses in folder successfully but unfortunately gets killed later. I'm unable to figure out the reason as to why this is happening.

Here's how I am running the tool:
image

Please help me understand why this is happening and how I can avoid this.
Thank you!

Inconsistent results

Used xnl with default config on yahoo.com but results were different most of the times.

python3 xnLinkFinder.py -i yahoo.com

Number of links it extracted each time

  • 1: 244
  • 2: 5
  • 3: 237
  • 4: 242
  • 5: 229
  • 6: 237
  • 7: 237
  • 8: 5
  • 9: 243
  • 10: 235

Input file specified two times

Hi!

Can't quite figure out what's going on here.
Potential links are 0, and the "Input file specified two times" comes up.

image

A more verbose output

image


When I specify the root domain manually, It still says "Input file specified two times" but yields couple of results.
image

Any idea?

SyntaxWarning: invalid escape sequence '\ '

When I install and run xnLinkFinder with python 3.12.0b4 I'm getting these warnings

root@c52dc151d463:/app/xnLinkFinder# python xnLinkFinder.py -h
/app/xnLinkFinder/xnLinkFinder.py:140: SyntaxWarning: invalid escape sequence '\.'
  DEFAULT_LINK_REGEX_FILES = "php|php3|php5|asp|aspx|ashx|cfm|cgi|pl|jsp|jspx|json|js|action|html|xhtml|htm|bak|do|txt|wsdl|wadl|xml|xls|xlsx|bin|conf|config|bz2|bzip2|gzip|tar\.gz|tgz|log|src|zip|js\.map"
/app/xnLinkFinder/xnLinkFinder.py:296: SyntaxWarning: invalid escape sequence '\ '
  write(colored("  \ / o-o  |      o-o  OO   O-o    o-o   o-O o-o o-o ", "green"))
/app/xnLinkFinder/xnLinkFinder.py:297: SyntaxWarning: invalid escape sequence '\ '
  write(colored("   o  |  | |    | |  | | \  |    | |  | |  | |-' |   ", "cyan"))
/app/xnLinkFinder/xnLinkFinder.py:298: SyntaxWarning: invalid escape sequence '\ '
  write(colored("  / \ o  o O---o| o  o o  o o    | o  o  o-o o-o o   ", "magenta"))
/app/xnLinkFinder/xnLinkFinder.py:389: SyntaxWarning: invalid escape sequence '\.'
  search = args.scope_filter.replace(".", "\.")
/app/xnLinkFinder/xnLinkFinder.py:396: SyntaxWarning: invalid escape sequence '\.'
  search = search.replace(".", "\.")
/app/xnLinkFinder/xnLinkFinder.py:627: SyntaxWarning: invalid escape sequence '\?'
  + ")(?:[\?|\/][^\"|']{0,}|))|([a-zA-Z0-9_\-]{1,255}\.(?:"
/app/xnLinkFinder/xnLinkFinder.py:629: SyntaxWarning: invalid escape sequence '\?'
  + ")(?:\?[^\"|^']{0,255}|)))(?:\"|'|\\n|\\r|\n|\r|\s|$)|(?<=^Disallow:\s)[^\$\n]*|(?<=^Allow:\s)[^\$\n]*|(?<= Domain\=)[^\";']*|(?<=\<)https?:\/\/[^>\n]*|(\"|\')([A-Za-z0-9_-]+\/)+[A-Za-z0-9_-]+(\.[A-Za-z0-9]{2,}|\/?(\?|\#)[A-Za-z0-9_\-&=\[\]]*)(\"|\')"
/app/xnLinkFinder/xnLinkFinder.py:852: SyntaxWarning: invalid escape sequence '\_'
  getSPACER(">>> Patience isn't your strong suit eh? ¯\_(ツ)_/¯"),
/app/xnLinkFinder/xnLinkFinder.py:1357: SyntaxWarning: invalid escape sequence '\%'
  newItem = re.sub("\%..", "", item)
/app/xnLinkFinder/xnLinkFinder.py:1358: SyntaxWarning: invalid escape sequence '\-'
  if len(newItem) > 2 and not newItem.startswith("_") and newItem.count("-") < 3 and re.match('^[A-Za-z0-9\-_]*[A-Za-z]+[A-Za-z0-9\-_]*$', newItem) and newItem.lower() not in lstStopWords: 
/app/xnLinkFinder/xnLinkFinder.py:1368: SyntaxWarning: invalid escape sequence '\-'
  if len(word) > 2 and re.match('^[A-Za-z0-9\-_]*[A-Za-z]+[A-Za-z0-9\-_]*$', word) and word.lower() not in lstStopWords:
/app/xnLinkFinder/xnLinkFinder.py:2410: SyntaxWarning: invalid escape sequence '\ '
  filePath = os.path.abspath(args.input).replace(" ", "\ ")
/app/xnLinkFinder/xnLinkFinder.py:2542: SyntaxWarning: invalid escape sequence '\s'
  """
/app/xnLinkFinder/xnLinkFinder.py:2558: SyntaxWarning: invalid escape sequence '\ '
  filePath = os.path.abspath(args.input).replace(" ", "\ ")
/app/xnLinkFinder/xnLinkFinder.py:2560: SyntaxWarning: invalid escape sequence '\s'
  cmd = "grep -Eo '^={3,4}\s?[0-9]+\s={10}$' --text " + filePath + " | wc -l"
/app/xnLinkFinder/xnLinkFinder.py:2600: SyntaxWarning: invalid escape sequence '\s'
  match = re.search("={3,4}\s?[0-9]+\s={10}", line)
/app/xnLinkFinder/xnLinkFinder.py:2638: SyntaxWarning: invalid escape sequence '\ '
  filePath = os.path.abspath(args.input).replace(" ", "\ ")
/app/xnLinkFinder/xnLinkFinder.py:2837: SyntaxWarning: invalid escape sequence '\s'
  match = re.search("={3,4}\s?[0-9]+\s={10}", firstLine)
/app/xnLinkFinder/xnLinkFinder.py:3009: SyntaxWarning: invalid escape sequence '\s'
  match = re.search("={3,4}\s?[0-9]+\s={10}", firstLine)
/app/xnLinkFinder/xnLinkFinder.py:3602: SyntaxWarning: invalid escape sequence '\.'
  help="RegEx for filtering purposes against found endpoints before output (e.g. /api/v[0-9]\.[0-9]* ). If it matches, the link is output.",
usage: xnLinkFinder.py [-h] [-i INPUT] [-o OUTPUT] [-op OUTPUT_PARAMS] [-owl OUTPUT_WORDLIST] [-ow] [-sp <domain/file>] [-spo] [-spkf]
                       [-sf <domain/file>] [-c COOKIES] [-H HEADERS] [-ra REGEX_AFTER] [-d DEPTH] [-p <integer>] [-x EXCLUDE] [-orig]
                       [-prefixed] [-xrel] [-t <seconds>] [-inc] [-u [...]] [-insecure] [-s429] [-s403] [-sTO] [-sCE] [-m <integer>]
                       [-mfs <integer>] [-rp REPLAY_PROXY] [-ascii-only] [-mtl <integer>] [--config CONFIG] [-nwlpl] [-nwlpw] [-nwlpm]
                       [-nwlc] [-nwlia] [-nwld] [-nwll] [-wlml WORDLIST_MAXLEN] [-swf STOPWORDS_FILE] [-nb] [-v] [-vv] [--version]

xnlLinkFinder (v4.1) - by @Xnl-h4ck3r

What is the correct python3 version to run the app?

提取参数问题

提取参数有很多不准确的,像这种是不是考虑一下下个版本优化
_0x48bdb1
_0x949cb2
_0x27e7c3
_0x1d9336

doesnt find links

greetings the tools for some reason doesn't seem to find any links from the js. it was tested with a link that has known URLs and endpoints but still it didn't find anything.
below is a sample screenshot
image
for comparison purposes here's sample output from linkfinder
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.