GithubHelp home page GithubHelp logo

vsemionov / django-blacklist Goto Github PK

View Code? Open in Web Editor NEW
20.0 3.0 10.0 34 KB

Blacklist users and hosts in Django. Automatically blacklist rate-limited clients.

License: MIT License

Python 100.00%

django-blacklist's Introduction

Django Blacklist

Blacklist users and hosts in Django. Automatically blacklist rate-limited clients.

Overview

Django Blacklist allows you to block specific users and IP addresses/networks from accessing your application. Clients can be blocked manually from the admin interface, or automatically after exceeding a request rate limit. Each blacklist rule is applied for a specific duration. The blacklist is very scalable and is applied without noticeable overhead for large numbers of rules.

Installation

To install the package, run:

$ pip install django-blacklist

Add the blacklist application to INSTALLED_APPS:

INSTALLED_APPS = [
    ...
    'blacklist'
]

Add the BlacklistMiddleware middleware after AuthenticationMiddleware:

MIDDLEWARE = [
    ...
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'blacklist.middleware.BlacklistMiddleware',
    ...
]

Apply the blacklist database migrations:

$ python manage.py migrate blacklist

Usage

You can manage the blacklist rules from the admin. Changes take effect after a configurable time, or when the server is restarted. A rule can target a user or an IP address. You can also target IP networks (ranges) by specifying the optional prefixlen field (number of network prefix bits). Each rule has a specific duration. After that time interval passes, the rule expires. When a request is blocked due to a matching rule:

  • Status 400 (bad request) is returned.
  • An error template is rendered. You can specify a custom one (see Settings below), or use the one for status 400.
  • A message is logged (warning from logger blacklist.middleware for custom templates, or error from logger django.security otherwise).

Removing Expired Rules

Expired rules are not automatically removed from the database. They can be cleaned up with the included management command trim_blacklist:

$ python manage.py trim_blacklist [-c <created_days>] [-e <expired_days>]

The options -c and -e specify the minimum ages of creation and expiry, respectively.

Automatic Blacklisting

Clients can be blacklisted automatically, after exceeding a specified request rate limit. This feature requires django-ratelimit.

First, rate-limit a view by applying the @ratelimit decorator. Make sure to set block=False. Then, blacklist rate-limited clients by adding the @blacklist_ratelimited decorator. Specify the blacklist duration. For example:

from datetime import timedelta
from django_ratelimit.decorators import ratelimit
from blacklist.ratelimit import blacklist_ratelimited

@ratelimit(key='user_or_ip', rate='50/m', block=False)
@blacklist_ratelimited(timedelta(minutes=30))
def index(request):
    ...

Automatic rules take effect immediately. If the request comes from an authenticated user, the rule will target that user. Otherwise, it will target their IP address.

@blacklist_ratelimited accepts two arguments: (duration, block=True).

  • duration can be a timedelta object, or a tuple of two separate durations (for user-based and IP-based rules).
  • block specifies if the request should be blocked immediately, or passed to the view.

Automatic rules will have a comment that contains the ID of the request, which triggered the creation of the rule, and the "request line". The request ID is added only if available. Django does not generate request IDs. For that purpose, you can install django-log-request-id.

Proxies and Client Addresses

By default, the client IP address is taken from the REMOTE_ADDR value of request.META. If your application server is behind one or more reverse proxies, this will usually be the address of the nearest proxy, and not the actual client address. To properly blacklist clients by IP address, you can configure Django Blacklist to use addresses from another source (see Settings below).

To actually obtain the proxied client addresses, you can use django-ipware. In this case, you can configure Django Blacklist to obtain client addresses from your function, which in turn calls django-ipware for the actual logic.

Alternatively, you can set REMOTE_ADDR from the X-Forwarded-For header in middleware, installed before Django Blacklist. However, keep in mind that this header can be forged to bypass the rate limits. To counter that, you can use the last address in that header (which should be set by your trusted reverse proxy). If you are behind two proxies, use the second to last address, and so on.

Settings

  • BLACKLIST_ENABLE - whether blacklisted clients should be blocked, and rate-limited clients should be blacklisted; default: True
  • BLACKLIST_RELOAD_PERIOD - how often to reload the blacklist, in seconds; default: 60
  • BLACKLIST_RATELIMITED_ENABLE - whether rate-limited clients should be automatically blacklisted; requires BLACKLIST_ENABLE; default: True
  • BLACKLIST_TEMPLATE - name of a custom error template to render to blocked clients; its context will contain request and exception; set to None to use the template for status 400; default: None
  • BLACKLIST_LOGGING_ENABLE - whether blocked requests should be logged (honored only if a custom error template is configured); default: True
  • BLACKLIST_ADDRESS_SOURCE - the source of client addresses; can be a key in request.META, a callable that receives the request object, or the dotted string path to such a callable; default: 'REMOTE_ADDR'

django-blacklist's People

Contributors

richardarpanet avatar vsemionov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

django-blacklist's Issues

AttributeError Format for duration arithmetic

Error:

Environment:


Request Method: GET
Request URL: http://127.0.0.1:8000/admin/

Django Version: 3.0.4
Python Version: 3.7.6
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django_countries',
 'languages',
 'blacklist',
 'taggit',
 'user',
 'model',
 'administrator']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'blacklist.middleware.blacklist_middleware']



Traceback (most recent call last):
  File "D:\anaconda3\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
    response = get_response(request)
  File "D:\anaconda3\lib\site-packages\blacklist\middleware.py", line 40, in middleware
    _load_blacklist()
  File "D:\anaconda3\lib\site-packages\blacklist\middleware.py", line 98, in _load_blacklist
    for rule in rules:
  File "D:\anaconda3\lib\site-packages\django\db\models\query.py", line 276, in __iter__
    self._fetch_all()
  File "D:\anaconda3\lib\site-packages\django\db\models\query.py", line 1261, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "D:\anaconda3\lib\site-packages\django\db\models\query.py", line 115, in __iter__
    for row in compiler.results_iter(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size):
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 1103, in results_iter
    results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 1138, in execute_sql
    sql, params = self.as_sql()
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 490, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup()
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 51, in pre_sql_setup
    self.setup_query()
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 42, in setup_query
    self.select, self.klass_info, self.annotation_col_map = self.get_select()
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 257, in get_select
    sql, params = self.compile(col)
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 422, in compile
    sql, params = node.as_sql(self, self.connection)
  File "D:\anaconda3\lib\site-packages\django\db\models\aggregates.py", line 88, in as_sql
    return super().as_sql(compiler, connection, **extra_context)
  File "D:\anaconda3\lib\site-packages\django\db\models\expressions.py", line 633, in as_sql
    arg_sql, arg_params = compiler.compile(arg)
  File "D:\anaconda3\lib\site-packages\django\db\models\sql\compiler.py", line 422, in compile
    sql, params = node.as_sql(self, self.connection)
  File "D:\anaconda3\lib\site-packages\django\db\models\expressions.py", line 448, in as_sql
    return DurationExpression(self.lhs, self.connector, self.rhs).as_sql(compiler, connection)
  File "D:\anaconda3\lib\site-packages\django\db\models\expressions.py", line 494, in as_sql
    sql, params = self.compile(self.rhs, compiler, connection)
  File "D:\anaconda3\lib\site-packages\django\db\models\expressions.py", line 484, in compile
    return connection.ops.format_for_duration_arithmetic(sql), params

Exception Type: AttributeError at /admin/
Exception Value: 'DatabaseOperations' object has no attribute 'format_for_duration_arithmetic'

Perhaps of note: I am using djongo

Adding setting that allows to override the header name that contains the real IP address

Hi there,

thanks a lot for this plugin! It has helped me a lot recently to reduce spam on my blog.

I'm hosting my app on Pythonanywhere. The issue is that they're changing the header of the requests slightly, so the real IP address of the client ends up under a different key, i.e. request.META.get('HTTP_X_REAL_IP'), see their docs.

I was thinking about adding a setting, e.g. BLACKLIST_REMOTE_ADDR, that would allow you to override the default header name (defaults to "REMOTE_ADDR"). Would you be happy to review a PR? Many thanks!

Adding proxy

Hi,

Can you describe with more detail, how to I pass the proxy server and add clients ip, not proxy ip?

Support for Custom User

When I try to run the migration I get:

blacklist.Rule.user: (fields.E301) Field defines a relation with the model 'auth.User', which has been swapped out.
        HINT: Update the relation to point at 'settings.AUTH_USER_MODEL'.

My settings.AUTH_USER_MODEL:
`AUTH_USER_MODEL = 'users.CustomUser' # Custom User Model

line 15 in blacklist/models.py needs updated to:
user = models.ForeignKey(settings.AUTH_USER_MODEL, null=True, blank=True, on_delete=models.CASCADE)

And from django.conf import settings needs added to imports.

I can do a PR if you prefer.

Disabling Logging

Hello, thanks for your work on the plugin! Would it be possible to add in a setting variable to disable logging? When blacklisted IPs hit our endpoints repeatedly it leads to some spam in our logs. It would be great if we could just reject ip's silently. Thanks very much,
Steve

AttributeError 'str' object has no attribute 'tzinfo'

Hello
I have error after adding one rule
id 1
created 2021-03-31 21:56:33.792587
updated 2021-03-31 22:31:44.223164
address 196.245.0.0
prefix 16
duration 7900000000
user_id NULL

python3.8/site-packages/blacklist/middleware.py in _load_blacklist
for rule in rules:

Variable | Value
addr_blacklist | {}
current_time | datetime.datetime(2021, 3, 31, 22, 32, 57, 493809, tzinfo=)
rules | Error in formatting: AttributeError: 'str' object has no attribute 'tzinfo'
until | Max(F(created) + F(duration))
user_blacklist | {}

Can you help me?

Access is denied to all users.

In the production environment, the guest user's IP address is detected as 127.0.0.1.
Therefore, access to the site is closed to all users. How can I block the guest user's IP address?

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.