unixfox / antibot-proxy Goto Github PK

View Code? Open in Web Editor NEW

17.0 17.0 2.0 44 KB

Proxy program designed to combat against basic bots that aren't a full browser.

Home Page: https://hub.docker.com/r/unixfox/antibot-proxy

JavaScript 98.01% Dockerfile 1.99%

bot-detection nodejs proxy

antibot-proxy's People

Contributors

Stargazers

Watchers

Forkers

hlfh

antibot-proxy's Issues

Only category: General showing results

Hello folks, SOS.

Not sure what I'm messing up in my setup but my SearXNG instance is only showing results for the general category. When I try to search for images, videos, news, etc on the GUI the buttons don't seem to work. Without the antibot-proxy, it works well.

https://searx.syncpundit.com/search?q=groot&language=en-US&time_range=&safesearch=0&theme=simple gives me a

Rate limit exceeded

Here's my config.toml:

COOKIE_NAME = 'searx'
ENDPOINT_NAME = 'searx.css'
HTML_TAG_REPLACE = '(<div id="pagination">)|(<head>)'
JAIL_PATH = '/home/searx-admin/jail'
MAX_RETRY = 5
PORT = 3030
TARGET = '127.0.0.1:4004'
TIMEOUT_LOAD = 120
WHITELIST = ['8.8.8.8', '1.1.1.1', '51.15.252.168']

[WHITELIST_PAGES]
GET = ['^/$', '^/autocompleter(.*)$', '^/favicon.ico$', '^/about$', '^/robots.txt$', '^/opensearch.xml$', '^/searx$']
POST = ['^/preferences$']

If there's anything I can do to help troubleshoot further please let me know

PS: the instance is public, only shared with friends and family

modularity

Separate each protection as a module. This will give the ability to easily add and remove protections in case of #6 for example.
And a dependency system so that a protection can't run before another.

verification page before serving the real content

Similar to the anti DDOS of cloudflare serve a verification page that makes the browser wait based on this method: https://github.com/kkuchta/css-only-chat#receiving-data then add HTML code for verifying if it's a real browser and finally if the browser reach the antibot resource redirect to the real content with a 302 or delete all the HTML code if possible then serve the real HTML content of the program proxied.
Store in a cookie or/and whitelist temporally the IP if it's a real browser.
Or if the client fail multiple times ban it.

community blacklist and whitelist

a community blacklist that everyone can register
and a whitelist enabled by default that everyone can submit
requires a github account to be able to submit to the blacklist and whitelist
blocklist and whitelist enabled by default

Auto update mechanism

An auto update mechanism with different channels:

stable
not stable

google (unexpected crash CAPTCHA required

Hi all,

google (unexpected crash CAPTCHA required

I got this error when search on page 3 or above, how to fix it?

https://github.com/unixfox/antibot-proxy
can antibot-proxy fix this error?

Thanks

Block non residential IP

An ability to detect non residential IPs and block it.
Resources:

https://www.spigotmc.org/resources/anti-vpn.58291/

0 : Detect if it's a bot after serving the real HTML code with a basic bot technique
1 : Detect if it's a bot before serving the real HTML code with a basic bot technique
2 : Detect if it's a bot before serving the real HTML code with an advanced technique like browser fingerprinting

block tor exit nodes

Use https://github.com/mahrud/caddy-altonions to distinguish each tor visitor instead of relying on exit nodes IP.
More information about the implementation here: https://blog.cloudflare.com/cloudflare-onion-service/

unixfox / antibot-proxy Goto Github PK

antibot-proxy's People

Contributors

Stargazers

Watchers

Forkers

antibot-proxy's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs