dkruchinin / sanic-prometheus Goto Github PK
View Code? Open in Web Editor NEWPrometheus metrics for Sanic, an async python web server
License: MIT License
Prometheus metrics for Sanic, an async python web server
License: MIT License
Good day,
Histogram metric already provides you with _count
counter [1]. Why to have a dedicated counter?
[1] https://prometheus.io/docs/concepts/metric_types/#histogram
Nice work on this. I like the direction that you are headed. I'll be honest, I have not looked too deeply into the code base yet, but have you put any thought into providing some sort of an API to providing custom metrics?
For example, it would be nice if I had some sort of a decorator, or something that would allow me to report back a custom metric on a particular endpoint. In particular, I am thinking about a way to report to prometheus the number of live sessions and websocket connections are open.
Since it is a chore to make the template manually, I don't know if there are any images that have been debugged. Can you provide references?If I did, I would thank your family!!!
When you reach a 404, it looks like the before_request_handler
is not called and thus triggers the following error in the after_request_handler
:
[2020-06-10 15:13:42 +0200] [17655] [ERROR] Exception occurred in one of response middleware handlers
Traceback (most recent call last):
File "/Users/optroodt/venvs/<snip>/lib/python3.7/site-packages/sanic/app.py", line 1017, in handle_request
request, response, request_name=name
File "/Users/optroodt/venvs/<snip>/lib/python3.7/site-packages/sanic/app.py", line 1315, in _run_response_middleware
_response = await _response
File "/Users/optroodt/venvs/<snip>/lib/python3.7/site-packages/sanic_prometheus/__init__.py", line 125, in before_response
metrics.after_request_handler(request, response, get_endpoint)
File "/Users/optroodt/venvs/<snip>/lib/python3.7/site-packages/sanic_prometheus/metrics.py", line 56, in after_request_handler
lat = time.time() - request['__START_TIME__']
File "/Users/optroodt/venvs/<snip>/lib/python3.7/site-packages/sanic/request.py", line 143, in __getitem__
return self.ctx.__dict__[key]
KeyError: '__START_TIME__'
Running:
There is no 1.8 version avaliable to install from pypi
prometheus-client
released 0.5.0 a few days ago, and it breaks the api.
Should probably limit the version in setup.py
to prometheus-client ~= 0.4.2
to install the latest-known-good version.
My error logs are getting filled with a
line 62, in after_request_handler
lat = time.time() - request['__START_TIME__']
KeyError: '__START_TIME__'
and I suspect it is some kind of conflict with sanic-limiter
by bohea (https://github.com/bohea/sanic-limiter). When I disable my limiter sanic-prometheus
handles fine.
I can fork and try to patch this but I thought I'd bring it to your attention first to see if you've had any problem.
Sample code to run with sanic-limiter
and sanic-prometheus
installed. Latest version of both from git (sanic-limiter is a bit out of date via pip). Run this and go to localhost:8000/home
then try localhost:8000/
. After going to an endpoint that will throw a 404 you'll see the prometheus error.
from sanic_prometheus import monitor
from sanic import Sanic, response, Blueprint
from sanic_limiter import Limiter, get_remote_address
app = Sanic()
test_bp = Blueprint('test')
limiter = Limiter(
app,
global_limits=['10 per second', '500 per day'],
key_func=get_remote_address)
@test_bp.route('/home', methods=['GET'])
async def home(request):
return response.json({'success': 'you are home'})
if __name__ == '__main__':
monitor(app).expose_endpoint()
app.blueprint(test_bp)
app.run(host='127.0.0.1', port=8000, access_log=True)
we're getting a deprecation warning:
/usr/local/lib/python3.9/site-packages/sanic/base.py:35: UserWarning: Setting variables on Sanic instances is deprecated and will be removed in version 21.9. You should change your Sanic instance to use instance.ctx.metrics instead.
It's better to use time.monotonic()
which is guaranteed to never decrease. time.time()
can go backwards due to, e.g., malfunctioning ntp service.
Is there a reason for prometheus-client~=0.5.0
in setup.py
? I've been running with 0.6.0
for a while now without problems. The current version is 0.7.1
however.
Can you please prepare new release to pypi?
Also, this is very strange, but version 0.1.4 cannot be loaded from pypi
Collecting sanic-prometheus==0.1.4
Could not find a version that satisfies the requirement sanic-prometheus==0.1.4 (from versions: 0.0.1, 0.1.0, 0.1.2, 0.1.3)
No matching distribution found for sanic-prometheus==0.1.4
Hi can you please prepare new release with #11 or above
Checked on sanic==0.6.0
.
After app finished, monitoring coroutine was forced finished with message.
Task was destroyed but it is pending!
task: <Task pending coro=<make_periodic_memcollect_task.<locals>.collector() running at /home/siredvin/.pyenv/pastereport/lib/python3.6/site-packages/sanic_prometheus/metrics.py:51> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fcb0b71c6d8>()]>>
I guess, in monitoring method should be added something like:
@app.listener('after_server_stop')
def after_stop(app, loop):
metric_collection_task.cancel()
@dkruchinin Will you support this repository? I can provide pull request for this issue.
Hello,
I get a KeyError: '__START_TIME__'
when I send an OPTIONS request, maybe the library should ignore the OPTIONS req. I'm also using it together with Sanic-Cors
.
Sanic no longer allows for item assignment in the request object.
I am using sanic==20.6.3 and sanic-prometheus==0.2.0
ERROR:sanic.root:Exception occurred while handling uri: 'http://localhost:8080/status/ok'
Traceback (most recent call last):
File "/home/test/.local/lib/python3.8/site-packages/sanic/app.py", line 906, in handle_request
response = await self._run_request_middleware(
File "/home/test/.local/lib/python3.8/site-packages/sanic/app.py", line 1265, in _run_request_middleware
response = await response
File "/home/test/.local/lib/python3.8/site-packages/sanic_prometheus/__init__.py", line 120, in before_request
metrics.before_request_handler(request)
File "/home/test/.local/lib/python3.8/site-packages/sanic_prometheus/metrics.py", line 52, in before_request_handler
request['__START_TIME__'] = time.time()
TypeError: 'Request' object does not support item assignment
Traceback (most recent call last):
File "/home/test/.local/lib/python3.8/site-packages/sanic/app.py", line 906, in handle_request
response = await self._run_request_middleware(
File "/home/test/.local/lib/python3.8/site-packages/sanic/app.py", line 1265, in _run_request_middleware
response = await response
File "/home/test/.local/lib/python3.8/site-packages/sanic_prometheus/__init__.py", line 120, in before_request
metrics.before_request_handler(request)
File "/home/test/.local/lib/python3.8/site-packages/sanic_prometheus/metrics.py", line 52, in before_request_handler
request['__START_TIME__'] = time.time()
TypeError: 'Request' object does not support item assignment
If I have an endpoint at /v2/ping
, it appears that all the metrics come through as just /v2
and there's no way to know if the request was for /v2/ping
or maybe /v2/pong
I'm facing this issue with latest sanic-prometheus. It happens when I accessed a non-existed page..
[ERROR] Exception occurred in one of response middleware handlers
Traceback (most recent call last):
File "/home/.../lib/python3.6/site-packages/sanic/app.py", line 977, in handle_request
request, response, request_name=name
File "/home/.../lib/python3.6/site-packages/sanic/app.py", line 1281, in _run_response_middleware
_response = await _response
File "/home/.../lib/python3.6/site-packages/sanic_prometheus/__init__.py", line 125, in before_response
metrics.after_request_handler(request, response, get_endpoint)
File "/home/.../lib/python3.6/site-packages/sanic_prometheus/metrics.py", line 56, in after_request_handler
lat = time.time() - request.ctx.__START_TIME__
AttributeError: 'types.SimpleNamespace' object has no attribute '__START_TIME__'
Running:
Ubuntu 20.04
Python 3.6.9
sanic 20.6.3
sanic-prometheus 0.2.1
If running Sanic in multi-worker mode (ie app.run(workers=...)
), it appears that we run into an issue documented in the prometheus_client README titled "Multiprocess Mode".
If I repeatedly refresh the /metrics end point in my dev environment after hitting some endpoints a few times, the output will "rotate" between the correct metrics and blank or underreported metrics. I'm guessing the underreported/blank metrics are from worker processes that haven't served requests yet.
I'm guessing multiprocess_mode
should be added to the various sanic-prometheus metrics.
I found that this way of prometheus metrics server starting, which code is
class _ThreadingSimpleServer(ThreadingMixIn, HTTPServer):
"""Thread per request HTTP server."""
def start_http_server(port, addr='', registry=REGISTRY):
"""Starts an HTTP server for prometheus metrics as a daemon thread"""
CustomMetricsHandler = MetricsHandler.factory(registry)
httpd = _ThreadingSimpleServer((addr, port), CustomMetricsHandler)
t = threading.Thread(target=httpd.serve_forever)
t.daemon = True
t.start()
is starting the server, which has memory leaks. I found it by profiling app with tracemalloc
library. This server is infinitely opening new threads. I think this behavior must be specified in the docs.
2017-12-27 13:35:38,132 asyncio ERROR Task was destroyed but it is pending!
task: <Task pending coro=<make_periodic_memcollect_task..collector() running at /usr/local/lib/python3.5/dist-packages/sanic_prometheus/metrics.py:42> wait_for=>
the error disappear when monitor(app).expose_endpoint() is not used
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.