singingwolfboy / flask-sse

Server-Sent Events for Flask

Flask SSE

A Flask extension for HTML5 server-sent events support, powered by Redis.

Example of sending events:

from flask import Flask
from flask_sse import sse

app = Flask(__name__)
app.config["REDIS_URL"] = "redis://localhost"
app.register_blueprint(sse, url_prefix='/stream')

@app.route('/send')
def send_message():
    sse.publish({"message": "Hello!"}, type='greeting')
    return "Message sent!"

To receive events on a webpage, use Javascript to connect to the event stream, like this:

var source = new EventSource("{{ url_for('sse.stream') }}");
source.addEventListener('greeting', function(event) {
    var data = JSON.parse(event.data);
    // do what you want with this data
}, false);

The full documentation for this project is hosted on ReadTheDocs.

flask-sse's Issues

Max Redis client connection issue

I have updated the max client connection to 90000 in the redis conf file and the ulimit is also set to 100000.
Previously the client connection count was set to 30000 so we used to get MaxClientConnection reached error.

Is there a way to identify if the client connection is closed in Flask and how to remove the idle connections?
Whenever the EventSource object is initiated it increments the client connection count.

flask+mod_wsgi+flask-sse is possible>?

I tried with websockets ended with incompatable with apache issue. Shall I use this with mod_wsgi?
Is gevent could run in apache_mod_wsgi?

Questions about proxying with nginx and keep-alives

Hey @singingwolfboy, I am hoping you can offer some advice on how to implement keep-alives; via ping and pong between the client and server. Here's the event-source RFC for ping and pong. I've come to the conclusion that I need to implement keep-alive's. But just in case I am asking for the wrong information here are some specifics.

I am getting a "504 gateway timeout error" when proxy-serving with nginx. When accessing the stream via gunicorn, I do not get a 504 timeout. So it's definitely on the nginx side. The 504 timeout happens on /stream and on endpoints like /stream?channel=pocoo. Here is the full nginx config I am using.

I've followed this serverfault answer for setting up the nginx configuration, which recommends dispatching the X-Accel-Buffering and Cache-Control headers for the sse endpoints. But even with this, I still get the 504 timeout.

@sse.after_request                                                                                            
def add_nginx_sse_headers(response):                                                                          
    response.headers['X-Accel-Buffering'] = 'no'                               
    response.headers['Cache-Control'] = 'no-cache'                             
    return response

At this point I'm not exactly sure what I'm doing wrong. I'm taking shots in the dark :) but I am glad to distill some of this into docs for the Flask-SSE (once I've figured it out). I would really like to hear your advice. Thanks!

Also I have tried setting keepalive_timeout 0; in the nginx config. I thought that would prevent the timeout, but no, I still get the 504 :( And that is what brought to implementing keep-alive's.

Streaming over HTTPS sends broken content

I don't know the full parameters of this, but I've been using flask-sse on GKE, with Gunicorn and Gevent workers. I've found that SSE over http works well, but over https gives fractured responses and doesn't push through fully.

frankxl's fork of this repo has a fix for this here: franklx@8030f48 - I don't know if there are any other implications from this fix, but it's resolved my issue.

503 Error

Edit: better to ask this way:

How can I run sse from a background worker on Heroku? If I have it as in the example, it will take up my enter web processing and block my app from responding to HTTP requests.

Empty response with gunicorn

how do we configure sse for dev env? i m able to send but get empty res on client

*var source = new EventSource("{{ url_for('sse.stream') }}");
* source.addEventListener('greeting', function (event) {
var data = JSON.parse(event.data);
$('#dropdown3').append(' f ');
}, false);

Why do I need redis?

This should be called flask_redis_sse instead.

Redis blocking connection issue

Problem -> We are implemented the SSE events on our prod app, We faced issue that if we hit multiple time more the 15 times application got stuck and didn't got any response.
Our Application have below config-
Nginx as Proxy Server
Twisted as wsgi to Run Flask App

Issue -> When i debugged this issue i find out that when we are calling the /stream endpoint it's tring to listen the Redis messages using below code
for pubsub_message in pubsub.listen()
here the pubsub.listen() is a blocking call until didn't got any message. In this case the Flask one thread got busy with this operation, if we hit multiple time all the flask app thread will be busy with the given operation. So the app will be unreachable in some time.

Solution -> While we are making connection to Redis we need to provide the socket_timeout to timeout the listen operation after certain time.
For more details go through here - redis/redis-py#225

I submitted a merge Request on this issue. Please find the link -
#13

Pushing notifications when an inactive users becomes active

Does flask-sse provide some kind of queueing for each client to receive notifications that they received when they are inactive?
Can you give me some insights on how to implement this mechanism?

Add headers to support nginx proxying

Hello @singingwolfboy ,

I use a setup where the SSE stream requests are poxied by nginx. This did not work until I added headers to the python response as described here. There is a very similar issue already closed in this repo: #3.

The important part, though, is that the headers must be set by the app server, not in nginx. I created a PR to show you how it could be solved in flask-sse.

Redis clients not disconnecting

Hello David,

I'm using your Flask extension and it has been super helpful so thank you for making it available!

I've been running into an issue however and was hoping you could help out. For some reason Redis clients keep filling up. Whenever someone starts listening for events it never disconnects. Only after I restart the Gunicorn instance running your Flask SSE extension do the clients disappear.

I'm running Flask SSE as follows:
gunicorn --workers 1 --worker-class gevent --bind unix:sse.sock -m 007 run_sse:app

Versions:

gunicorn==19.7.0
Flask==0.12
gevent==1.1.1
Flask-SSE==0.2.1
redis==2.10.5

I run behind Nginx with the following configuration:

proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 24h;
proxy_pass http://sse_server;
chunked_transfer_encoding off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

And here is the Flask SSE code:

from flask import Flask
from flask_sse import sse


app = Flask(__name__)
app.config["REDIS_URL"] = "redis://localhost"
app.register_blueprint(sse, url_prefix='/listen')

I'm running a separate synchronous Gunicorn worker where I render the webpages where the client side code is found. This separate Flask instance also publishes to Redis. I use your extension only to listen for events. Is this the reason that Redis is filling up?

Or am I doing something wrong in my Nginx configuration?

Hoping you can shed some light on the situation.

Thank you for you time.

How to get rid of redis dependency?

The redis pubsub seems to be just one way of handling the message stream. See WolfgangFahl/pyFlaskBootstrap4#17 for my attempt to use a non redis dependent vresion of flask-sse by mixing with https://maxhalford.github.io/blog/flask-sse-no-deps/

https://github.com/WolfgangFahl/pyFlaskBootstrap4/tree/main/tests has test_sse.py and test_sse_messages.py as a start as well as https://github.com/WolfgangFahl/pyFlaskBootstrap4/blob/main/fb4/sse_bp.py for a start of the blueprint which basically copies code from this repo.

I'd rather get the redis dependency resolved right here by some kind of ducktyping the pubsub part so the implementation can be choosen. To e.g. use a simple solution like https://maxhalford.github.io/blog/flask-sse-no-deps/
https://github.com/MaxHalford/flask-sse-no-deps as replacement

How to generate continous event stream

I am trying to generate a continous event stream with this library.

I get this to work by using an endpoint which has to be called by the client which looks like:

@app.route('/api/get/statusupdate')
def status_update():
    controller = TestController()
    while True:
        sse.publish(controller.get_status(), type='greeting')
        time.sleep(5)

I am configuring the plugin with this:

app.config["REDIS_URL"] = "redis://localhost"
app.register_blueprint(sse, url_prefix='/stream')

Now I can call /stream in the client to listen to events, but I need to call /api/get/statusupdate first which is a workaround. Is it possible to do this withiout calling /api/get/statusupdate and directly output some data on the event endpoint?

Does Flask_sse work cross origin?

DOing Flask (port 3000)/React (port 5000 using Vite dev server). Not using the Jinja templates, so SendEvent is hard coded and seems to fire. sse.publish is never "recorded" by Redis (using monitor) and the stream hangs pending?

Possible Enhancement to Disable SSE in Development

David, following up on our conversation about the possibility of disabling the extension to quickly switch back to using the flask development server.

What do you think about using a config variable to control it? I haven't looked at the source code much yet, but maybe there's a simple way to toggle initialization of the extension by looking for the existence of a particular app config variable? It would have to gracefully handle the blueprint too, since it's most likely there will be some url_fors in an app's templates and that would throw BuildErrors.

Let me know what you think about this direction. I'd be happy to offer a PR if you agree.

Unable to run: Error Is Redis running?

I have a system running on IP: 192.168.247.129. I have redis running on that. Upon doing ps aux | grep redis I get:

redis 50690 0.2 0.1 50152 3808 ? Ssl 10:55 0:01 /usr/bin/redis-server 192.168.247.129:6379

(The IP is different since I've put that in /etc/redis/redis.conf)

I tried running the quickstart code and its not working.

Adding event to subscribe type message from Redis

For me Flask-SSE does not work with React hooks SSE (https://github.com/samouss/react-hooks-sse).

It does work, however, if the code form (declined) PR#22 (#22) is added.

How to run the tests?

Tried:

python --version
Python 3.8.6

pytest

Leading to ModuleNotFoundError: No module named 'flask_sse' which is an issue some 200k others also have see:
https://stackoverflow.com/questions/10253826/path-issue-with-pytest-importerror-no-module-named-yadayadayada

The resolution seems to be:

python -m pytest tests/

which leads to:

E       fixture 'mocker' not found

Could you please add a command line test script and or a description how to run the tests?

Proper Design for Flask-Login Support?

Sorry this is more like a question rather than an issue report -- I'm just out of idea who to turn to when my SO question ( https://stackoverflow.com/questions/54379538/how-to-support-flask-sse-access-control ) is not getting much view / answer.

Basically I need to support both public and private SSE ( unlike in my SO question, I finally decided to use channel private.<user_id> instead of route ). Right now I'm monkey-patching Flask-SSE with --

        @stream_with_context
        def generator():
            if channel.startswith('private.'): # private area
                channel_user_id = channel.replace('private.', '').lower()
                if not (current_user and current_user.is_authenticated and 
                        current_user.get_id().lower() == channel_user_id):
                    raise StopIteration
            else:
                for message in self.messages(channel=channel):
                    yield str(message)

I'm not sure if it's a good way or even a right way. I'd really appreciate any pointer.

sse channel as celeby backend

I want use Celeby to handle tasks with long processing time

Here I read about backend
http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#keeping-results

Can I configure this backend to send results to flask-sse channel?

Thanks for any help.

Too many open files due to generator not closing connection.

I find a problem when the client opens a stream and the server doesn't push anything, it will cause connection to stuck and won't close, then too many open files error happened.

So I edit some code to send some dummy messages (like ping) to prevent that happen.

And here's my workaround :

        def generator():
            pubsub = self.redis.pubsub()
            pubsub.subscribe(channel)
            try:
                while True:
                    message = pubsub.get_message()
                    if message == None:
                        yield "event:ping\ndata:{}\n\n"
                        time.sleep(0.4)
                    elif message['type'] == 'message':
                        msg_dict = json.loads(message['data'])
                        yield str(Message(**msg_dict))
            except GeneratorExit:
                pass
            except:
                traceback.print_exc()
            finally:
                pubsub.unsubscribe(channel)

Any idea how to replace redis with AWS SQS ?

Hello,

I'm wondering if it's possible to replace redis with AWS SQS.

Thanks for any idea.

Closing connections?

I've had no trouble implementing SSE using this library however I am wondering how to get the client to close the connections. Apache server-status shows many open connections and this is holding open many processes. I beleive that over time these open connections clog up the availiable processes.

Srv	PID	Acc	M	CPU	SS	Req	Conn	Child	Slot	Client	Protocol	VHost	Request
0-0	7653	0/22/22	W	62.77	15499	0	0.0	1.47	1.47	XXX.XXX.82.252	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images150 HTTP/1.1
1-0	7654	1/15/15	W	2.75	16422	0	7.8	1.95	1.95	XXX.XXX.82.252	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images150 HTTP/1.1
2-0	7655	0/15/15	W	1.27	16418	0	0.0	0.52	0.52	XXX.XXX.82.252	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images150 HTTP/1.1
3-0	7656	0/23/23	W	244.03	16089	0	0.0	0.16	0.16	XXX.XXX.82.252	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images150 HTTP/1.1
4-0	7657	0/16/16	W	123.45	16416	0	0.0	0.34	0.34	XXX.XXX.82.252	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images150 HTTP/1.1
5-0	7734	0/15/15	W	138.78	16089	0	0.0	0.81	0.81	XXX.XXX.82.252	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images150 HTTP/1.1
6-0	7735	1/31/31	W	2.81	72	0	3.8	1.38	1.38	XXX.XXX.42.137	http/1.1	EXAMPLE.COM:443	GET /commands?channel=images152 HTTP/1.1

Many are from the same IP address as well meaning any time a user goes to a page that subscribes to an EventSource it takes up another slot.

I added a 'beforeunload' section to try to close the eventsource when the user navigates away from te page.

    $(window).on('beforeunload', function () {
        event_source.close();
        event_source = null;
    });

But the close doesn't seem to do anything

sse.before_request not working

Hi, I've imported default blueprint sse from flask_sse

after

@sse.before_request
def check_access():
    if request.args.get("channel") == "analytics" and not g.user.is_admin():
        abort(403)

Running the app with flask and accessing URL at http://localhost:5000/events sse.before request will not be run, don't know why. I've checked with debugger.

Has this something to do with flask:see or with blueprints in general?

Listening to Flask SSE events with a Python client

Hello there,

Thank you for flask-sse which is a very nice library. I face an issue when I try to connect to a Flask SSE server from a Python client. When I run the following command, the process is blocked while it should keep on executing command after running the GET request:

import requests
response = requests.get("http://serverdomain/events", stream=True)
print("do something with the response")

What I do wrong with Flask-sse?

Here is my server code (spawn via gunicorn):

app = Flask(__name__)
app.config["REDIS_URL"] = redis_url
app.register_blueprint(sse, url_prefix='/events')

---

By the way I have another question. Javascript clients or curl are disconnected after 30s. Do you have any recommendation on how to handle this. Should I reconnect the client each time the error event is fired?

Want to send SSE with RQ worker, how to configure?

Can I send SSEs using an RQ worker running in a different docker container?

My code looks like this right now:

# app_factory.py
def create_app(app_name=__name__, **kwargs):

	app = Flask(app_name, template_folder="./templates")

	#* SSE Configuration
	app.config['REDIS_URL'] = REDIS_URI_SSE

# RQ Worker function
from flask_sse import sse
def emit_markers(map_id):
	sse.publish(marker_data, type="markers", channel=f"{content_id}")
	sse.publish(status_data, type="status", channel=f"{content_id}")

Explanation: Any single client will be connected to a content_id and will listen for markers and status, as these are both needed to populate the frontend page.
The worker will do some CPU heavy processing and generate the marker_data and status_data periodically on a cron schedule. The worker would need to independently send this to all the clients connected on that particular channel.

Where do I put the API endpoint? I don't really see a reason for it here? But on the client side I need to provide a value to the EventSource declaration:

const sse = new EventSource('/api/v1/sse');

Any help will be much appreciated.

subscribe channel name based on session

I need send personal events to user session

flask-sse/flask_sse.py

Line 150 in a5c6afe

channel = request.args.get('channel') or 'sse'

How can I do it?

Please delete