jazzband / django-revproxy

Reverse Proxy view that supports all HTTP methods, Diazo transformations and Single Sign-On.

Welcome to django-revproxy

A simple reverse proxy using Django. It allows to use Django as a reverse Proxy to HTTP requests. It also allows to use Django as an authentication Proxy.

Documentation available at http://django-revproxy.readthedocs.org/

Features

• Proxies all HTTP methods: HEAD, GET, POST, PUT, DELETE, OPTIONS, TRACE, CONNECT and PATCH
• Copy all http headers sent from the client to the proxied server
• Copy all http headers sent from the proxied server to the client (except hop-by-hop)
• Basic URL rewrite
• Sets the http header REMOTE_USER if the user is logged in Django
• Sets the http headers X-Forwarded-For and X-Forwarded-Proto
• Handles redirects
• Few external dependencies
• Apply XSLT transformation in the response (requires Diazo)

Dependencies

• django >= 3.0
• urllib3 >= 1.12
• diazo >= 1.0.5 (optional)
• lxml >= 3.4, < 3.5 (optional, but diazo dependency)

Install

pip install django-revproxy

django-revproxy's Issues

Read Docs page failed to load css

Remove support to Python < 3 and Django < 2

Since Python 2 is no longer supported I think this now becomes a good option. Any objections?

Rewrite subdomain to subdirectory

Hello!

I've got origin that is as follows:

origin.tld/first
origin.tld/second

My goal is to find a reverse proxy that would translate following:

first.tld into origin.tld/first
and second.tld/page into origin.tld/second/page

Is this possible at the moment with this package?

Modify GET query parameters

I needed a way of modify the GET query parameters before forwarding the request to upstream and I created a subclass of ProxyView that accepts a query_rewrite field for doing this.

If you think this can be useful I can arrange the code for a pull request.

Add support to Django 1.9

Diazo conditional rules are not working (only when used with django-revproxy)

Redirects not handled correctly

301 Redirects by the upstream server causes the proxied url to redirect to the upstream path, rather than the proxied path + upstream path.

Add REMOTE_USER_DATA header

Add a custom header and allow developers do customize the data that will be sent on it.

The use of self.unicode_content doubles memory usage

Better logging system

Add more logging messages, our response and upstream response.

Iterate over HTTP response instead of load it all in memory

X-Forwarded-For/Forwarded header

Does it make sense to send (either by default or enabled with an option) a forward header?

https://en.wikipedia.org/wiki/X-Forwarded-For

Capture broken responses and return django's error page instead

Use standard error message and log errors using logging.error.

Proxy behind proxy

Hi there,

Does it use the environment http_proxy and https_proxy variables? I mean should it use?
I've tried to use in a closed (AWS) environment (no outgoing connections without a squid proxy) and looks like doesn't work.

Absolute prefix - how?

Hi,
I've been trying to find a simple solution to use revproxy to proxy our blog (http://blog.example.com) onto a subdirectory of our django app. I need to do 2 things to make it work.

1. set a meta header for canonical content e.g.
2. prepend all relative links/paths with '/blog/'
3. revproxy all requests to /blog to blog.example.com

From research I think 1 & 2 can be done using diazo. (completely new to me). 1) using a xslt rule

but 2) seems to be achievable by using http://docs.diazo.org/en/latest/compiler.html#absolute-prefix But I just can't work out how to tie this together top django-revproxy.

Is this supported? or do I need to use my own transform mechanism (beautiful soup etc?)

Too many redirects

I'm running a django server on a reaspberry pi.

I have a service (transmission-daemon) listening on port 9091 and I want it to be accessible through the django server (as a url path) . After configuration when trying to access the url path the i get ERR_TOO_MANY_REDIRECTS error in my browser, and looking at the logs it does seems like it repeatedly redirects. when i used nginx for this proxy set-up it worked fine.

django-revproxy version: 0.10.0
django version: 3.0.4
python version: 3.8

viewes.py:

class TransmissionProxy(ProxyView):
    upstream = 'http://X.X.X.X:9091'

urls.py

urlpatterns = [
    path('admin/', admin.site.urls),
    url(r'^transmission(?P<path>.*)$', views.TransmissionProxy.as_view(), name="proxy_test"),
    url(r'^$', views.Home.as_view(), name='home'),
]

django logs:

ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:27] "GET /transmission/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:37] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:37] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:37] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:38] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:39] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:39] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:39] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:39] "GET /transmission/web/ HTTP/1.1" 301 31
ProxyView created
Normalizing response headers
Checking for invalid cookies
[13/Jun/2020 10:41:39] "GET /transmission/web/ HTTP/1.1" 301 31

Cookies become invalid if they have an equals symbol in their value.

utils.py converts cookie strings to dict. It splits cookie key and value using split funciton. In this line.

cookie_dict['key'], cookie_dict['value']=cookie_parts[0].split('=')

but note that if the cookie value or key contains an equal sign, it will get splitted into more than two parts thereby invalidating the cookie.

"'WSGIRequest' object has no attribute 'user'" if add_remote_user without AuthenticationMiddleware

We are developing a small module that will be will in turn be used in a full django application. We had been using revproxy successfully, but realized that user information was not coming through. Adding add_remote_user would fix that, but with that addition our tests errored with:

'WSGIRequest' object has no attribute 'user'

The problem is that the module tests did not use AuthenticationMiddleware, so user was not added to request and this line fails:

if self.add_remote_user and self.request.user.is_active:

For now, we've added AuthenticationMiddleware in our code, but would you accept a PR to change that line to

if self.add_remote_user and hasattr(self.request, 'user') and self.request.user.is_active:

or do you see a problem with that?

0.9.14 to pypi

Submit please new version to pypi
And requests raise warning, because urllib3<1.17 in setup.py

Incorrect Changelog

0.9.15 does not support Django 3 and it wasn't released on 2019-12-03.

https://pypi.org/project/django-revproxy/#history

da78089

Connection pool is full, discarding connection:

I am trying to create a reverse proxy to a internal website. It works for the most part however one endpoint I get the following warning:

WARNING [2019-01-29 15:46:27,893] [none] urllib3.connectionpool: Connection pool is full, discarding connection: 127.0.0.1

I assume somewhere in the code I can increase the pool size limit but do I need to make this change?

built-in POST request calling mistaken QueryDict method

I'm in the project that evolves the use of Colab. We're working on integrating Redmine on Colab, however I stumbled on a bug. Step-by-step as follows:

1. Send a POST request to colab, that contains an array like "...arr[]=a&arr[]=b..."
2. Colab will receive the request
3. Colab will redirect the request to Redmine machine, through django/revproxy plugin.
4. Redmine receives the array, but only the final value is passed on: "...arr[]=b..."

Trying to figure it out, I forged a POST request as follows:
----------------------- POST -------------------
POST http://localhost:8000/redmine
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20100101 Firefox/24.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: csrftoken=DXJj7yQH6rt4n39rSSJsjBnGSyzaiB23; utc_offset=180
Connection: keep-alive
Content-Length: 15

Content: arr[]=a&arr[]=b

Colab uses a View object that should be implemented by every running service under Colab. This View is an extension of a file that is at /usr/local/lib/python2.7/site-packages/revproxy/views.py.

Here's the important part:

1.   if request.POST:
   

2.        if 'multipart/form-data' in request_headers.get('Content-Type', ''):
   

3.             proxy_request.add_data(request_payload)
   

4.         else:
   

5.             post_data = request.POST.items()  
   

6.             proxy_request.add_data(urllib.urlencode(post_data))
   

request.POST is a QueryDict, from Django, its content is as follows:
<QueryDict: {u'arr[]': [u'a', u'b']}>, it means, values 'a' and 'b' are still set in "request.POST"

The problem is at line 5, in order to get POST values there's a call to QueryDict.items(), which by its documentation is a last-value logic. Having that in mind, "post_data" will receive "...arr[]=b..." only, which lacks of value "a" in that dictionary:
this is what actually is being passed => post_data = [(u'arr[]', u'b')]

QueryDict documentation shows that the method returning all values from a dictionary is
QueryDict.lists(). Should "lists()" be the most appropriate method in this case?

admin in path in reverse proxy url in not working

Hello Everyone,

I am have following url in reverse proxy
re_path(r'demo/(?P<path>.*)$',` ProxyView.as_view(upstream='http://127.0.0.1:5000/'))

I am running flask application at 5000 port. django application run at 8000

Following url works:
http://localhost:8000/demo/admin/role/

However, following url doesn't work
http://localhost:8000/demo/admin/ and it redirect to http://localhost:8000/admin/ with following url The current path, admin/, didn't match any of these.

Does any one know how to fix this issue ?

dispatch() takes exactly 3 arguments (2 given)

Currently all the docs use

urlpatterns = patterns('',
    url(r'^(?P<path>.*)$', TestProxyView.as_view()),
)

Tested working ok with this style of urlpatterns BUT as of Django 1.10 (currently 1.9.7) this will be removed.

RemovedInDjango110Warning: django.conf.urls.patterns() is deprecated and will be removed in Django 1.10. Update your urlpatterns to be a list of django.conf.urls.url() instances instead.
  url(r'^(?P<path>.*)$', TestProxyView.as_view()),

Trying the new stlye of url from django docs

urlpatterns = [
    url(r'^admin/', admin.site.urls),
    url(r'^$', TestProxyView.as_view()),
    url(r'createsession/', TestProxyView.as_view(), name='session'),

Gives the following error

Environment:


Request Method: GET
Request URL: http://192.168.1.105:8000/createsession/

Django Version: 1.9.7
Python Version: 2.7.11
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'revproxy',
 'Portal']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback:

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  149.                     response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  147.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python2.7/dist-packages/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)

Exception Type: TypeError at /createsession/
Exception Value: dispatch() takes exactly 3 arguments (2 given)

urllib2 is not thread safe

See: http://stackoverflow.com/questions/5825151/are-urllib2-and-httplib-thread-safe

Upgrade urllib3 dependency

Could the urllib3 dependency be upgraded? This package depends on urllib3 <= version 1.16. However, 1.16 is now 12 months old and the latest version is 1.21.1. Depending on an older version of urllib3 makes it difficult to use django-revproxy in a project that uses other packages that depend on urllib3; for example, the latest version of python-requests depends on urllib3 version 1.21: https://github.com/requests/requests/blob/master/setup.py#L47

deploy on pythonanywhere.com,not working

because pythonanywhere limit free accout visit Internet with a white list,need to set up proxy-server(proxy.server:3128) in django

and my site report Error:
HTTPConnectionPool(host='www.google.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<revproxy.connection.RevProxyHTTPConnection object at 0x7f909e457208>: Failed to establish a new connection: [Errno 101] Network is unreachable',))

But I use urllib3 in teminal to visit www.google.com ,it is working.

How to solve this problem ?
I already setup http_proxy in .bashrc:

export http_proxy='http://proxy.server:3128'
export https_proxy='http://proxy.server:3128'

dispatch() missing 1 required positional argument: 'path'

urlpatterns = [
url('api/test', ProxyView.as_view(upstream='http://localhost:8080/'))
]

Respect X-Diazo-Off header

https://github.com/plone/diazo/blob/master/lib/diazo/wsgi.py#L20

Use wsgiref to check hop-by-hop headers

https://docs.python.org/2/library/wsgiref.html

Allow transformations on compressed response

If a response comes compressed we should be able to decompress it, transform and compress it again

StreamingHTTPResponse and hardcoded threshold

Hi, I was wondering if you had a special reason to hardcode a very low threshold for returning a StreamingHTTPResponse instead of HTTPResponse.

To my understanding, the Django docs generally discourage the use of StreamingHTTPResponse:
https://docs.djangoproject.com/en/2.0/ref/request-response/#streaminghttpresponse-objects

How about making this user configurable and add the option not to use StreamingHTTPResponse at all?
I'd put together a PR for that in case you support the idea.

Accept multiple upstreams and balance load

Add tests for django1.8

Release new version

Please release new version with the django 3 support.

I see the necessary fix is in the master already.

thank you very much!

Release 0.10.0 as old version does not allow urllib3 > 1.17

It seem django-revprox is not getting user with rest_framework_simplejwt

I tried to load user in custom _created_proxy_response method which inherent from ProxyView. In which i am trying to add custom logic to if auth is there or not. But it seems to be not working. I am able to see token but not username. Is anyone know how to fix this ?

From,
Zen

Write documentation (sphinx)

HTTP 403 Forbidden when post ajax request

"POST /BOE/portal/1905100745/InfoViewAppActions/ajaxUre/ajaxRequest HTTP/1.1" 403 1278

Need to handle base URL

Hello,

Let's say you want to have http://myproxy/toto.html redirect to http://proxied/toto.html. Then a simple url(r'^(?P<path>.*)$', ProxyView.as_view(upstream='http://proxied/')) will do the trick. And the static files in the page will also work properly. It will, for example refer to http://myproxy/css/default.css properly.

Now you may want to have a base URL because this proxy will work for more than one site, for example. You would want something like this:

url(r'^app1/(?P<path>.*)$', ProxyView.as_view(upstream='http://hostapp1/')),
url(r'^app2/(?P<path>.*)$', ProxyView.as_view(upstream='http://hostapp2/')),

But it doesn't work because all static files will forget about the base url app1 or app2. So the page http://myproxy/app1/toto.html will refer to http://myproxy/css/default.css instead of http://myproxy/app1/css/default.css.

Please add the possibility to handle this in revproxy.

Thanks,
Timok

Release new version

Could you please release a new version to pypi?

Remove six from deps

Django includes six so it's better if we just use django's.

Pinned URL version

Hi there,

thanks very much for django-revproxy — it's really useful!

We've just run into an issue, though — we noticed that in this commit 0904ed4 you pinned the version of urllib3. We've now gone past that version, which is obviously causes conflicts.

Would it be possible to unpin urllib3?

Thanks very much. Yours,

Stephen Betts.

What is the lastest Django version does this lib support?

Hi there,

Does anyone have the idea of what the latest Django version does this library support? Many thanks!

Cheers

Implement caching using Django cache API

Adding example in documentation Quickstart for latest django version.

Here at https://django-revproxy.readthedocs.io/en/latest/quickstart.html

Remove Empty Request Headers

it seems that django's request.META, adds many http headers with empty values if they does not exist in client request. e.g. client send a request without Content-Type header, but request.META contains a CONTENT_TYPE key with empty string value.
i think revproxy should not send this headers to upstreams, as it does not make sense to send Content-Type: '' to a web server

It's possible to proxy Websocket?

Hi, i'm trying to proxy behind django Node-Red, a node-based programming tool for IoT.

This application, uses heavily Webosocket in order to communicate with remote MQTT brokers, Websockets, etc...

Actually, i've tried to do some tests and the proxied http content works good, but the WS content it's not working, and django-channels complains that the route was not found...i'm not sure if this package supports websocket proxying, can someone will solve this doubt?

Thanks!

https server not work

I requested https service, he reported an error.

HTTPSConnectionPool(host='***, port=443): Max retries exceeded with 
url: /** (Caused by SSLError(SSLError("bad handshake:
Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

I read the source code and made the following changes, it work, but I want to know, is there an official way to complete the https request?

class ProxyView(ProxyView):
    upstream = 'https://****/'

    def __init__(self, *args, **kwargs):
        super(ProxyView, self).__init__(*args, **kwargs)
        self.http = urllib3.PoolManager(cert_reqs='CERT_REQUIRED', ca_certs=certifi.where())
  

Allow to configure if should pass request headers and body

Similar to nginx proxy_pass_request_headers and proxy_pass_request_body:

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass_request_headers