ashcrow / flask-track-usage

Basic metrics tracking for the Flask framework.

flask-track-usage

Basic metrics tracking for the Flask framework. This focuses more on ip addresses/locations rather than tracking specific users pathing through an application. No extra cookies or javascript is used for usage tracking.

• Read the latest docs
• Read 2.0.0 docs
• Read 1.1.1 docs

Similar Projects

• https://github.com/srounet/Flask-Analytics

flask-track-usage's Issues

Include X-Forwarded-For as well as remote_addr

Created from #16

"Since it is easy to forge an X-Forwarded-For field the given information should be used with care. The last IP address is always the IP address that connects to the last proxy, which means it is the most reliable source of information. X-Forwarded-For data can be used in a forward or reverse proxy scenario.

Just logging the X-Forwarded-For field is not always enough as the last proxy IP address in a chain is not contained within the X-Forwarded-For field, it is in the actual IP header. A web server should log BOTH the request's source IP address and the X-Forwarded-For field information for completeness."

Why not log both instead of guessing? Would love to hear feedback on this.

Adding pagination to get_usage?

What do you think about adding a page argument to the get_usage method. That way the load on the DB will be less and people can add paginated monitors.

import issue with flask-3.0.0

Traceback (most recent call last):
...
from flask_track_usage import TrackUsage
File "/opt/homebrew/lib/python3.11/site-packages/flask_track_usage/init.py", line 43, in
from flask import _request_ctx_stack, g
ImportError: cannot import name '_request_ctx_stack' from 'flask' (/opt/homebrew/lib/python3.11/site-packages/flask/init.py)

The above import error is caused by removing the deprecated code got removed from flask-3.0.0:

pallets/flask#5223

_app_ctx_stack and _request_ctx_stack are removed as places to store custom data in favor of g. The deprecation period was already extended from 2.3.

Can't import "sumVisitors"

I'm trying to use the sumVisitors hook but I get the error when I try to import it:

ImportError: cannot import name 'sumVisitors' from 'flask_track_usage.summarization'

Any idea?

Thanks.

[BUG] [Proposal] SQL hook has no attribute 'set_up'

SQL Hooks (and MongoHooks as far as I have seen) are splitted into 2 different methods:

def sumUrl_set_up(*args, **kwargs):
        tables = ["url_hourly", "url_daily", "url_monthly"]
        create_tables(tables, **kwargs)

def sumUrl(**kwargs):
    if not _check_environment(**kwargs):
        return
    if not _check_postgresql(**kwargs):
        raise NotImplementedError("Only PostgreSQL currently supported")
        return
    [...]

However in the Storage class method __init__ it is called as the following:

flask-track-usage/src/flask_track_usage/storage/__init__.py

Line 66 in 629ad1c

hook.set_up(**kwargs)

Two solutions came to my mind:

1. Change Storage.__init__ to call <hook>_set_up function instead.
2. Change how hooks are created to be classes instead methods

Upgrade SQL schema from 1.x -> 2.x

Provide either an automatic upgrade OR a good error message pointing to documentation on how to upgrade schema for moving from 1.x to 2.x.

TRACK_USAGE_INCLUDE_OR_EXCLUDE_VIEWS requirement not clear

I opened this to see if we can clarify the use of the variable TRACK_USAGE_INCLUDE_OR_EXCLUDE_VIEWS.

What is the real purpose of this? We have include and exclude methods that explicitly specify what methods or blueprints to include or exclude. Not sure I understand how to correctly use this. Perhaps this can be clarified in the docs? Or can we simplify the API to just work with include and exclude methods only?

Is it possible to get accept_languages instead of user_agent.language

Thank you so much for creating this great package!

When I'm using it in most of the cases the ua_language is empty.

I think it's because of the usage of request.user_agent.language instead of request.accept_languages.
Browsers don't so often set the language but the accepted lanuages should be there nearly every time.

A change here may be a very great thing.

Thank you so much!

Fix database lock up in SQLStorage

@ashcrow Having used the SQLStorage for a while now, I have had to fix some issues I encountered with database lock up. Wondering if you are interested in taking the changes.

SQL table 'flask_usage.path' length (32) is too short

My website path is usually longer than 32 characters as I use uuid4 to encode an object id. Would you be able to increase this to 128 or even 256?

Add test case for more of the SQLAlchemy backends

I have only tested the SQLStorage with SQLite so far. Write test cases for other backends as well.

Spring clean before 2.x release

Remove complexity/out of date comments/etc.. before releasing.

Speculative features for (much) later versions

These are just thoughts for future releases; in no particular order. I figured I'd document them somewhere. I'm not going to work on any of them right now :).

• Use threading for storage so that a slow DB connection does not slow down Flask.
• Modify the Mongo updates (for summaries) so that the increment is more atomic; should scale better.
• Add Writers for feeding data much more advanced analytics products:
  • Matomo (via API or MySQL)
  • Sawmill (via API)
  • AWStats, which is done via a Apache Access Log, which leads us to....
• Add ApacheAccessLogWriter, which writes a standard apache log to a text file
• Add a HTTP-Status-Code summary, to see 404s etc.
• Add summary for GeoLocation
• Put sumBasic back in (does the basic 5 as a single line.)

Move datetime to timestamp conversion in an OS-safe way

I noticed you use 'date': int(now.strftime('%s')) in your TrackUsage.after_request() method. Though this version of converting datetime to timestamp is not supported officially. I actually get errors when I run in Windows.

I was thinking of moving to, import time; time.mktime(now.timetuple()).

double tracking

I've not been able to hunt down the exact reason, but it appears that I'm getting "double" entries for every web query. Even using command-line wget on a specific page, it showing two entries in the tracking table and two calls into the library: one "extremely short" one. Usually with a speed of 0.04s or less. And one entry that is more representative of the actual time to download the page. This might be a quirk of Flask, but it is inflating the numbers.

Any ideas what might be causing this? Is the same procedure being called before and after the request?

Verify docs are updated

Verify the docs are updated and deployed for 1.1.0.

Unittests for sql backend failing

Fix it.

Release 2.0.0

Merge, tag, and release flask-track-usage 2.0.0.

Adding write-time aggregation functions

Typically, in a web site, the usage data is written lightly and as fast as possible to a log. Then, a secondary, process (and possibly a secondary server) does statistical analysis on that data. This totally make sense for storage in a traditional database such as SQL and certainly for an apache-style text log.

Philosophically, however, MongoDB (and related NoSQL) databases can take a different approach. They are designed for scalablity using a variety of techniques: including an emphasis of read-optimization at the expense of write-optimization. The lack of normalization, for example, makes it expensive to update (write) certain data because such updates might have to occur across many documents. But in exchange for that, a read of any one document in a collection need never reference another document because all the important information is already gathered.

Sorry be so windy, but I'm wanting to justify my crazy idea. :) And that is this:

Rather than just write a single document to a collection on each page response, also allow aggregate updates on other documents at the same time.

For example, it is common in web log analysis to record the number of visits to each page over certain periods of time. Say hourly, daily, monthly. So, when flask-track-usage records a response, it could also upsert the url/datetime/period documents corresponding to it with incremented totals. In this example, it would update 3 additional documents.

This would be implemented as an option of course. There would be scenarios where such aggregate work would be a bad idea or pointless.

One possiblity is to have it done as a post-storage function call. For example:

def myCrazySummationUtility(data):
    # here is where I do all my extra stuff with the dictionary
    # contained in 'data'
    ....

t = TrackUsage(app, PrintStorage(post=myCrazySummation))

Thoughts?

Incompatible with Redis 3

RedisStorage is not compatible with newer versions of Redis. When trying to store the data in Redis an redis.exceptions.DataError is thrown (see this line. This is because Redis 3.0 only accepts keys and values as bytes, strings or numbers as stated here.

Also it would be nice, if it would be possible to set up Redis with a connection url instead of using host and port. Many apps store their Redis connection like that and platforms like Scalingo also provide only the url as an env var.

[Idea] Possibility to exclude url path, not only views

I wanted to use TRACK_USAGE_INCLUDE_OR_EXCLUDE_VIEWS = 'exclude' to track every view except the admin view, however it tracked too the requests for static files (which I don't want to track), creating a lot of entries in the database.

I tried to exclude it with

tracker.exclude("/static")

But when requesting static files, TrackUsage.before_request -> self.app.view_functions.get(ctx.request.endpoint)

flask-track-usage/src/flask_track_usage/__init__.py

Line 109 in 629ad1c

view_func = self.app.view_functions.get(ctx.request.endpoint)

returns a lambda, which makes impossible (or at least I could find how to replicate) exclude those routes.

This could be very handy for semi-large/large applications where is faster just exclude 1 view than include all the other views.

I have seen that ctx.request.path returns the path (/, /static, /admin, etc), maybe checking both possibilities, views and routes could be a solution for this?

Release 1.0.0

Release 1.0.0 code to pypi and update pythonhosted documentation.

This has been moved to 1.0.0 because it has some backwards incompat changes.

user_agent field as a <user_agent> object instead of a string?

Python 3.5.2
flask 0.12.2

I am trying to save the log in a file as OutputWriter(output=open('page_view.log','a'), transform=lambda s: str(s)). In the output file I find that user_agent shows as 'user_agent': <UserAgent 'chrome'/74.0.3729.169>
It should be fine for storage backends, but it's a little bit problematic for an output file. (i.e json.dumps get TypeError: <UserAgent 'chrome'/74.0.3729.169> is not JSON serializable)

For the user_agent field:

flask-track-usage/src/flask_track_usage/__init__.py

Line 160 in cceb0d2

'user_agent': ctx.request.user_agent,

Is it better like this?

     'user_agent': ctx.request.user_agent.string,

or

     'user_agent': {
        'browser':ctx.request.user_agent.browser,    
        'language':ctx.request.user_agent.language,
        'platform':ctx.request.user_agent.platform,
        'version':ctx.request.user_agent.version,
        }

Use the flask-sqlalchemy metadata instead of creating a new one

I found a small issue with the way the MetaData is constructed in the sql.py. Would like to fix this if you are still taking pull requests!

Unify get_usage results between storage backends.

Currently there is no spec for deciding the results returned by get_usage. Create a unifying response object/spec that can be used for all storage backends.

(See #6)

When using app.config['TRACK_USAGE_USE_FREEGEOIP'] = True on localhost cannot write to mongoldb

I believe when writting to MongoDB one cannot have empty strings but the default values of localhost bring something like:

{
"businessName" : "",
"businessWebsite" : "",
"city" : "",
"continent" : "",
"country" : "",
"countryCode" : "",
"ipName" : "",
"ipType" : "Residential",
"isp" : "",
"lat" : "",
"lon" : "",
"org" : "",
"query" : "127.0.0.1",
"region" : "",
"status" : "success"
}

Causing issues when testing the capabilities.

Warn on upcoming changes

Since we are going to be releasing at least one backwards incompatible change give a heads up on the doc site.

influxdb storage

Create influxdb storage.

Flask-Track-Usage cannot be installed due to a versionConflict issue.

Since 1/9/2023, I have been experiencing an issue when deploying a web application, of which the "Flask-Track-Usage" is one of the dependencies. During the deployment process, I am repeatedly encountering a failure at the step of installing this library, as shown in the output from the deployment via VS Code below. It appears that the issue is related to a version conflict, although all of the dependencies prior to this library were version-locked and there were no issues until yesterday. I would be very grateful for any assistance or advice in resolving this issue.

11:24:33 AM tlbu0: Starting deployment... 11:24:37 AM tlbu0: Creating zip package... 11:25:41 AM tlbu0: Zip package size: 114 MB 11:25:45 AM tlbu0: Fetching changes. 11:25:46 AM tlbu0: Cleaning up temp folders from previous zip deployments and extracting pushed zip file /tmp/zipdeploy/6381c077-ced4-486e-84c2-d342c83765f6.zip (109.68 MB) to /tmp/zipdeploy/extracted 11:26:10 AM tlbu0: Updating submodules. 11:26:11 AM tlbu0: Preparing deployment for commit id '2dbd1d48-8'. 11:26:11 AM tlbu0: PreDeployment: context.CleanOutputPath False 11:26:11 AM tlbu0: PreDeployment: context.OutputPath /home/site/wwwroot 11:26:11 AM tlbu0: Repository path is /tmp/zipdeploy/extracted 11:26:12 AM tlbu0: Running oryx build... 11:26:12 AM tlbu0: Command: oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform python --platform-version 3.9 -p virtualenv_name=antenv --log-file /tmp/build-debug.log -i /tmp/8daf32fc4bbb7d4 --compress-destination-dir | tee /tmp/oryx-build.log 11:26:14 AM tlbu0: Operation performed by Microsoft Oryx, https://github.com/Microsoft/Oryx 11:26:14 AM tlbu0: You can report issues at https://github.com/Microsoft/Oryx/issues 11:26:14 AM tlbu0: Oryx Version: 0.2.20220812.1, Commit: cdf6b1bef165d05b94830e963646495967d938f4, ReleaseTagName: 20220812.1 11:26:14 AM tlbu0: Build Operation ID: |6Hk8mpxbdvw=.571f5390_ 11:26:14 AM tlbu0: Repository Commit : 2dbd1d48-8e7b-4105-8556-33be450f2243 11:26:14 AM tlbu0: Detecting platforms... 11:26:16 AM tlbu0: Detected following platforms: 11:26:16 AM tlbu0: nodejs: 14.19.1 11:26:16 AM tlbu0: python: 3.9.16 11:26:16 AM tlbu0: Version '14.19.1' of platform 'nodejs' is not installed. Generating script to install it... 11:26:16 AM tlbu0: Version '3.9.16' of platform 'python' is not installed. Generating script to install it... 11:26:18 AM tlbu0: Using intermediate directory '/tmp/8daf32fc4bbb7d4'. 11:26:18 AM tlbu0: Copying files to the intermediate directory... 11:26:35 AM tlbu0: Done in 18 sec(s). 11:26:35 AM tlbu0: Source directory : /tmp/8daf32fc4bbb7d4 11:26:35 AM tlbu0: Destination directory: /home/site/wwwroot 11:26:35 AM tlbu0: Downloading and extracting 'nodejs' version '14.19.1' to '/tmp/oryx/platforms/nodejs/14.19.1'... 11:26:40 AM tlbu0: Downloaded in 5 sec(s). 11:26:40 AM tlbu0: Verifying checksum... 11:26:40 AM tlbu0: Extracting contents... 11:26:44 AM tlbu0: performing sha512 checksum for: nodejs... 11:26:44 AM tlbu0: Done in 9 sec(s). 11:26:44 AM tlbu0: Downloading and extracting 'python' version '3.9.16' to '/tmp/oryx/platforms/python/3.9.16'... 11:26:46 AM tlbu0: Downloaded in 2 sec(s). 11:26:46 AM tlbu0: Verifying checksum... 11:26:47 AM tlbu0: Extracting contents... 11:26:52 AM tlbu0: performing sha512 checksum for: python... 11:26:55 AM tlbu0: Done in 11 sec(s). 11:26:55 AM tlbu0: Python Version: /tmp/oryx/platforms/python/3.9.16/bin/python3.9 11:26:55 AM tlbu0: Creating directory for command manifest file if it does not exist 11:26:55 AM tlbu0: Removing existing manifest file 11:26:57 AM tlbu0: Python Virtual Environment: antenv 11:26:57 AM tlbu0: Creating virtual environment... 11:27:10 AM tlbu0: Activating virtual environment... 11:27:10 AM tlbu0: Running pip install... 11:27:41 AM tlbu0: [17:27:12+0000] Collecting requests==2.22.0 11:27:41 AM tlbu0: [17:27:12+0000] Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB) 11:27:41 AM tlbu0: [17:27:12+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.0/58.0 KB 132.0 kB/s eta 0:00:00 11:27:41 AM tlbu0: [17:27:12+0000] Collecting requests_html==0.10.0 11:27:41 AM tlbu0: [17:27:12+0000] Downloading requests_html-0.10.0-py3-none-any.whl (13 kB) 11:27:41 AM tlbu0: [17:27:12+0000] Collecting cssmin==0.2.0 11:27:41 AM tlbu0: [17:27:12+0000] Downloading cssmin-0.2.0.tar.gz (3.2 kB) 11:27:41 AM tlbu0: [17:27:13+0000] Preparing metadata (setup.py): started 11:27:41 AM tlbu0: [17:27:13+0000] Preparing metadata (setup.py): finished with status 'done' 11:27:41 AM tlbu0: [17:27:13+0000] Collecting Werkzeug==2.0.0 11:27:41 AM tlbu0: [17:27:13+0000] Downloading Werkzeug-2.0.0-py3-none-any.whl (288 kB) 11:27:41 AM tlbu0: [17:27:13+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.1/288.1 KB 2.9 MB/s eta 0:00:00 11:27:41 AM tlbu0: [17:27:14+0000] Collecting WTForms==2.3.1 11:27:41 AM tlbu0: [17:27:14+0000] Downloading WTForms-2.3.1-py2.py3-none-any.whl (169 kB) 11:27:41 AM tlbu0: [17:27:14+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.1/169.1 KB 2.6 MB/s eta 0:00:00 11:27:41 AM tlbu0: [17:27:15+0000] Collecting SQLAlchemy==1.3.9 11:27:41 AM tlbu0: [17:27:15+0000] Downloading SQLAlchemy-1.3.9.tar.gz (6.0 MB) 11:27:41 AM tlbu0: [17:27:16+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 4.1 MB/s eta 0:00:00 11:27:41 AM tlbu0: [17:27:18+0000] Preparing metadata (setup.py): started 11:27:41 AM tlbu0: [17:27:20+0000] Preparing metadata (setup.py): finished with status 'done' 11:27:41 AM tlbu0: [17:27:20+0000] Collecting Flask==2.0.3 11:27:41 AM tlbu0: [17:27:20+0000] Downloading Flask-2.0.3-py3-none-any.whl (95 kB) 11:27:41 AM tlbu0: [17:27:20+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.6/95.6 KB 887.9 kB/s eta 0:00:00 11:27:41 AM tlbu0: [17:27:20+0000] Collecting jinja2==3.0.3 11:27:41 AM tlbu0: [17:27:20+0000] Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB) 11:27:41 AM tlbu0: [17:27:20+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.6/133.6 KB 2.2 MB/s eta 0:00:00 11:27:41 AM tlbu0: [17:27:21+0000] Collecting click>=8.0.1 11:27:41 AM tlbu0: [17:27:21+0000] Downloading click-8.1.3-py3-none-any.whl (96 kB) 11:27:41 AM tlbu0: [17:27:21+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 1.1 MB/s eta 0:00:00 11:27:42 AM tlbu0: [17:27:21+0000] Collecting Flask_Assets==2.0 11:27:42 AM tlbu0: [17:27:21+0000] Downloading Flask_Assets-2.0-py3-none-any.whl (8.5 kB) 11:27:42 AM tlbu0: [17:27:21+0000] Collecting flask-login==0.5.0 11:27:42 AM tlbu0: [17:27:21+0000] Downloading Flask_Login-0.5.0-py2.py3-none-any.whl (16 kB) 11:27:42 AM tlbu0: [17:27:22+0000] Collecting Flask_RESTful==0.3.9 11:27:42 AM tlbu0: [17:27:22+0000] Downloading Flask_RESTful-0.3.9-py2.py3-none-any.whl (25 kB) 11:27:42 AM tlbu0: [17:27:23+0000] Collecting Flask-Track-Usage==2.0.0 11:27:42 AM tlbu0: [17:27:23+0000] Downloading Flask-Track-Usage-2.0.0.tar.gz (37 kB) 11:27:42 AM tlbu0: [17:27:23+0000] Preparing metadata (setup.py): started 11:27:42 AM tlbu0: [17:27:39+0000] Preparing metadata (setup.py): finished with status 'error' 11:27:42 AM tlbu0: error: subprocess-exited-with-error 11:27:42 AM tlbu0:
11:27:42 AM tlbu0: × python setup.py egg_info did not run successfully. 11:27:42 AM tlbu0: │ exit code: 1 11:27:42 AM tlbu0: ╰─> [14 lines of output] 11:27:42 AM tlbu0: Traceback (most recent call last): 11:27:42 AM tlbu0: File "", line 2, in 11:27:42 AM tlbu0: File "", line 34, in 11:27:42 AM tlbu0: File "/tmp/pip-install-9o0d_pdw/flask-track-usage_cbe6cf81d7a2455593cfdbd19c2e2fad/setup.py", line 38, in 11:27:42 AM tlbu0: setup( 11:27:42 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/init.py", line 152, in setup 11:27:42 AM tlbu0: _install_setup_requires(attrs) 11:27:42 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/init.py", line 147, in _install_setup_requires 11:27:42 AM tlbu0: dist.fetch_build_eggs(dist.setup_requires) 11:27:42 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs 11:27:42 AM tlbu0: resolved_dists = pkg_resources.working_set.resolve( 11:27:42 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/pkg_resources/init.py", line 777, in resolve 11:27:42 AM tlbu0: raise VersionConflict(dist, req).with_context(dependent_req) 11:27:42 AM tlbu0: pkg_resources.ContextualVersionConflict: (sphinxcontrib.applehelp 1.0.3 (/tmp/pip-install-9o0d_pdw/flask-track-usage_cbe6cf81d7a2455593cfdbd19c2e2fad/.eggs/sphinxcontrib.applehelp-1.0.3-py3.9.egg), Requirement.parse('sphinxcontrib-applehelp'), {'sphinx'}) 11:27:42 AM tlbu0: [end of output] 11:27:42 AM tlbu0:
11:27:42 AM tlbu0: note: This error originates from a subprocess, and is likely not a problem with pip. 11:27:42 AM tlbu0: error: metadata-generation-failed 11:27:42 AM tlbu0: × Encountered error while generating package metadata. 11:27:42 AM tlbu0: ╰─> See above for output. 11:27:42 AM tlbu0: note: This is an issue with the package mentioned above, not pip. 11:27:42 AM tlbu0: hint: See above for details. 11:27:42 AM tlbu0: WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available. 11:27:42 AM tlbu0: You should consider upgrading via the '/tmp/8daf32fc4bbb7d4/antenv/bin/python -m pip install --upgrade pip' command. 11:27:42 AM tlbu0: "2023-01-10 17:27:41"|ERROR|[17:27:12+0000] Collecting requests==2.22.0 11:27:42 AM tlbu0: [17:27:12+0000] Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB) 11:27:42 AM tlbu0: [17:27:12+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.0/58.0 KB 132.0 kB/s eta 0:00:00 11:27:42 AM tlbu0: [17:27:12+0000] Collecting requests_html==0.10.0 11:27:42 AM tlbu0: [17:27:12+0000] Downloading requests_html-0.10.0-py3-none-any.whl (13 kB) 11:27:42 AM tlbu0: [17:27:12+0000] Collecting cssmin==0.2.0 11:27:42 AM tlbu0: [17:27:12+0000] Downloading cssmin-0.2.0.tar.gz (3.2 kB) 11:27:42 AM tlbu0: [17:27:13+0000] Preparing metadata (setup.py): started 11:27:42 AM tlbu0: [17:27:13+0000] Preparing metadata (setup.py): finished with status 'done' 11:27:42 AM tlbu0: [17:27:13+0000] Collecting Werkzeug==2.0.0 11:27:43 AM tlbu0: [17:27:13+0000] Downloading Werkzeug-2.0.0-py3-none-any.whl (288 kB) 11:27:43 AM tlbu0: [17:27:13+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.1/288.1 KB 2.9 MB/s eta 0:00:00 11:27:43 AM tlbu0: [17:27:14+0000] Collecting WTForms==2.3.1 11:27:43 AM tlbu0: [17:27:14+0000] Downloading WTForms-2.3.1-py2.py3-none-any.whl (169 kB) 11:27:43 AM tlbu0: [17:27:14+0000] ━━━━━━━���━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.1/169.1 KB 2.6 MB/s eta 0:00:00 11:27:43 AM tlbu0: [17:27:15+0000] Collecting SQLAlchemy==1.3.9 11:27:43 AM tlbu0: [17:27:15+0000] Downloading SQLAlchemy-1.3.9.tar.gz (6.0 MB) 11:27:43 AM tlbu0: [17:27:16+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 4.1 MB/s eta 0:00:00 11:27:43 AM tlbu0: [17:27:18+0000] Preparing metadata (setup.py): started 11:27:43 AM tlbu0: [17:27:20+0000] Preparing metadata (setup.py): finished with status 'done' 11:27:43 AM tlbu0: [17:27:20+0000] Collecting Flask==2.0.3 11:27:43 AM tlbu0: [17:27:20+0000] Downloading Flask-2.0.3-py3-none-any.whl (95 kB) 11:27:43 AM tlbu0: [17:27:20+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.6/95.6 KB 887.9 kB/s eta 0:00:00 11:27:43 AM tlbu0: [17:27:20+0000] Collecting jinja2==3.0.3 11:27:43 AM tlbu0: [17:27:20+0000] Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB) 11:27:43 AM tlbu0: [17:27:20+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.6/133.6 KB 2.2 MB/s eta 0:00:00 11:27:43 AM tlbu0: [17:27:21+0000] Collecting click>=8.0.1 11:27:43 AM tlbu0: [17:27:21+0000] Downloading click-8.1.3-py3-none-any.whl (96 kB) 11:27:43 AM tlbu0: [17:27:21+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 1.1 MB/s eta 0:00:00 11:27:43 AM tlbu0: [17:27:21+0000] Collecting Flask_Assets==2.0 11:27:43 AM tlbu0: [17:27:21+0000] Downloading Flask_Assets-2.0-py3-none-any.whl (8.5 kB) 11:27:43 AM tlbu0: [17:27:21+0000] Collecting flask-login==0.5.0 11:27:43 AM tlbu0: [17:27:21+0000] Downloading Flask_Login-0.5.0-py2.py3-none-any.whl (16 kB) 11:27:43 AM tlbu0: [17:27:22+0000] Collecting Flask_RESTful==0.3.9 11:27:43 AM tlbu0: [17:27:22+0000] Downloading Flask_RESTful-0.3.9-py2.py3-none-any.whl (25 kB) 11:27:43 AM tlbu0: [17:27:23+0000] Collecting Flask-Track-Usage==2.0.0 11:27:43 AM tlbu0: [17:27:23+0000] Downloading Flask-Track-Usage-2.0.0.tar.gz (37 kB) 11:27:43 AM tlbu0: [17:27:23+0000] Preparing metadata (setup.py): started 11:27:43 AM tlbu0: [17:27:39+0000] Preparing metadata (setup.py): finished with status 'error' 11:27:43 AM tlbu0: error: subprocess-exited-with-error 11:27:43 AM tlbu0:
11:27:43 AM tlbu0: × python setup.py egg_info did not run successfully. 11:27:43 AM tlbu0: │ exit code: 1 11:27:43 AM tlbu0: ╰─> [14 lines of output] 11:27:43 AM tlbu0: Traceback (most recent call last): 11:27:43 AM tlbu0: File "", line 2, in 11:27:43 AM tlbu0: File "", line 34, in 11:27:43 AM tlbu0: File "/tmp/pip-install-9o0d_pdw/flask-track-usage_cbe6cf81d7a2455593cfdbd19c2e2fad/setup.py", line 38, in 11:27:43 AM tlbu0: setup( 11:27:43 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/init.py", line 152, in setup 11:27:43 AM tlbu0: _install_setup_requires(attrs) 11:27:43 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/init.py", line 147, in _install_setup_requires 11:27:43 AM tlbu0: dist.fetch_build_eggs(dist.setup_requires) 11:27:43 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs 11:27:43 AM tlbu0: resolved_dists = pkg_resources.working_set.resolve( 11:27:44 AM tlbu0: File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/pkg_resources/init.py", line 777, in resolve 11:27:44 AM tlbu0: raise VersionConflict(dist, req).with_context(dependent_req) 11:27:44 AM tlbu0: pkg_resources.ContextualVersionConflict: (sphinxcontrib.applehelp 1.0.3 (/tmp/pip-install-9o0d_pdw/flask-track-usage_cbe6cf81d7a2455593cfdbd19c2e2fad/.eggs/sphinxcontrib.applehelp-1.0.3-py3.9.egg), Requirement.parse('sphinxcontrib-applehelp'), {'sphinx'}) 11:27:44 AM tlbu0: [end of output] 11:27:44 AM tlbu0:
11:27:44 AM tlbu0: note: This error originates from a subprocess, and is likely not a problem with pip. 11:27:44 AM tlbu0: error: metadata-generation-failed 11:27:44 AM tlbu0: × Encountered error while generating package metadata. 11:27:44 AM tlbu0: ╰─> See above for output. 11:27:44 AM tlbu0: note: This is an issue with the package mentioned above, not pip. 11:27:44 AM tlbu0: hint: See above for details. 11:27:44 AM tlbu0: WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available. 11:27:44 AM tlbu0: You should consider upgrading via the '/tmp/8daf32fc4bbb7d4/antenv/bin/python -m pip install --upgrade pip' command. | Exit code: 1 | Please review your requirements.txt | More information: https://aka.ms/troubleshoot-python 11:27:44 AM tlbu0: /bin/bash -c "oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform python --platform-version 3.9 -p virtualenv_name=antenv --log-file /tmp/build-debug.log -i /tmp/8daf32fc4bbb7d4 --compress-destination-dir | tee /tmp/oryx-build.log ; exit $PIPESTATUS " 11:27:44 AM tlbu0: Generating summary of Oryx build 11:27:44 AM tlbu0: Parsing the build logs 11:27:45 AM tlbu0: Found 1 issue(s) 11:27:45 AM tlbu0: Build Summary : 11:27:45 AM tlbu0: =============== 11:27:45 AM tlbu0: Errors (1) 11:27:45 AM tlbu0: 1. [17:27:12+0000] Collecting requests==2.22.0\n[17:27:12+0000] Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)\n[17:27:12+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.0/58.0 KB 132.0 kB/s eta 0:00:00\n[17:27:12+0000] Collecting requests_html==0.10.0\n[17:27:12+0000] Downloading requests_html-0.10.0-py3-none-any.whl (13 kB)\n[17:27:12+0000] Collecting cssmin==0.2.0\n[17:27:12+0000] Downloading cssmin-0.2.0.tar.gz (3.2 kB)\n[17:27:13+0000] Preparing metadata (setup.py): started\n[17:27:13+0000] Preparing metadata (setup.py): finished with status 'done'\n[17:27:13+0000] Collecting Werkzeug==2.0.0\n[17:27:13+0000] Downloading Werkzeug-2.0.0-py3-none-any.whl (288 kB)\n[17:27:13+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.1/288.1 KB 2.9 MB/s eta 0:00:00\n[17:27:14+0000] Collecting WTForms==2.3.1\n[17:27:14+0000] Downloading WTForms-2.3.1-py2.py3-none-any.whl (169 kB)\n[17:27:14+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 169.1/169.1 KB 2.6 MB/s eta 0:00:00\n[17:27:15+0000] Collecting SQLAlchemy==1.3.9\n[17:27:15+0000] Downloading SQLAlchemy-1.3.9.tar.gz (6.0 MB)\n[17:27:16+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 4.1 MB/s eta 0:00:00\n[17:27:18+0000] Preparing metadata (setup.py): started\n[17:27:20+0000] Preparing metadata (setup.py): finished with status 'done'\n[17:27:20+0000] Collecting Flask==2.0.3\n[17:27:20+0000] Downloading Flask-2.0.3-py3-none-any.whl (95 kB)\n[17:27:20+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 95.6/95.6 KB 887.9 kB/s eta 0:00:00\n[17:27:20+0000] Collecting jinja2==3.0.3\n[17:27:20+0000] Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)\n[17:27:20+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.6/133.6 KB 2.2 MB/s eta 0:00:00\n[17:27:21+0000] Collecting click>=8.0.1\n[17:27:21+0000] Downloading click-8.1.3-py3-none-any.whl (96 kB)\n[17:27:21+0000] ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 1.1 MB/s eta 0:00:00\n[17:27:21+0000] Collecting Flask_Assets==2.0\n[17:27:21+0000] Downloading Flask_Assets-2.0-py3-none-any.whl (8.5 kB)\n[17:27:21+0000] Collecting flask-login==0.5.0\n[17:27:21+0000] Downloading Flask_Login-0.5.0-py2.py3-none-any.whl (16 kB)\n[17:27:22+0000] Collecting Flask_RESTful==0.3.9\n[17:27:22+0000] Downloading Flask_RESTful-0.3.9-py2.py3-none-any.whl (25 kB)\n[17:27:23+0000] Collecting Flask-Track-Usage==2.0.0\n[17:27:23+0000] Downloading Flask-Track-Usage-2.0.0.tar.gz (37 kB)\n[17:27:23+0000] Preparing metadata (setup.py): started\n[17:27:39+0000] Preparing metadata (setup.py): finished with status 'error'\n error: subprocess-exited-with-error\n \n × python setup.py egg_info did not run successfully.\n │ exit code: 1\n ╰─> [14 lines of output]\n Traceback (most recent call last):\n File "", line 2, in \n File "", line 34, in \n File "/tmp/pip-install-9o0d_pdw/flask-track-usage_cbe6cf81d7a2455593cfdbd19c2e2fad/setup.py", line 38, in \n setup(\n File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/init.py", line 152, in setup\n _install_setup_requires(attrs)\n File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/init.py", line 147, in _install_setup_requires\n dist.fetch_build_eggs(dist.setup_requires)\n File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs\n resolved_dists = pkg_resources.working_set.resolve(\n File "/tmp/8daf32fc4bbb7d4/antenv/lib/python3.9/site-packages/pkg_resources/init.py", line 777, in resolve\n raise VersionConflict(dist, req).with_context(dependent_req)\n pkg_resources.ContextualVersionConflict: (sphinxcontrib.applehelp 1.0.3 (/tmp/pip-install-9o0d_pdw/flask-track-usage_cbe6cf81d7a2455593cfdbd19c2e2fad/.eggs/sphinxcontrib.applehelp-1.0.3-py3.9.egg), Requirement.parse('sphinxcontrib-applehelp'), {'sphinx'})\n [end of output]\n \n note: This error originates from a subprocess, and is likely not a problem with pip.\nerror: metadata-generation-failed\n\n× Encountered error while generating package metadata.\n╰─> See above for output.\n\nnote: This is an issue with the package mentioned above, not pip.\nhint: See above for details.\nWARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.\nYou should consider upgrading via the '/tmp/8daf32fc4bbb7d4/antenv/bin/python -m pip install --upgrade pip' command.
11:27:45 AM tlbu0: - Next Steps: Please review your requirements.txt 11:27:45 AM tlbu0: - For more details you can browse to https://aka.ms/troubleshoot-python 11:27:45 AM tlbu0: Warnings (0) 11:27:46 AM tlbu0: Deployment Failed. deployer = Push-Deployer deploymentPath = ZipDeploy. Extract zip. Remote build. 11:28:00 AM tlbu0: Deployment failed.

Verify user testing before release

Ask @gouthambs to help test before releasing 1.1.0.

GeoIP url function needing update

The current default for GeoIP is:

app.config['TRACK_USAGE_FREEGEOIP_ENDPOINT'] = 'http://freegeoip.net/json/'

which, according to ipstack.com, is being deprecated. It will stop working July 2018. Fortunately, they are keeping up the free service. But it is now requiring a login to get an API key:

http://api.ipstack.com/134.201.250.155?access_key=YOUR_ACCESS_KEY

What I'd like to do is generate a PR to update this (and the docs).

I'm thinking of having it make use a .format wildcard in the form of:

app.config['TRACK_USAGE_FREEGEOIP_ENDPOINT'] = 'http://api.ipstack.com/{ip}?access_key=YOUR_ACCESS_KEY'

and then updating the documentation to state that the config MUST be updated so that the user can set his/her API key.

Should the SQL Storage be executing db.create_all()?

From #22 (comment) by @gouthambs

Another thing is I feel the tables should not be created by this storage class.
Explicit invocation of db.create_all() should take care of that. Thats a small
change in usage.

This is consistent with how Flask-SQLAlchemy models just create the table
instances in the meta, but the explicit invocation of db.create_all() creates
the tables.

My initial thought is it does make sense to execute db.create_all() if the tables to not already exist. If they do exist the plugin should step back and let the admin/programmer do what needs to be done.

Right now if a user passes in a db instance they could easily execute db.create_all() when they are ready. However, if they are using conn_str method of setting up they wouldn't have access to do so.

However, I'm open to changing it as long as we can do it in a way that doesn't force the SQL Storage to be somewhat special. We could add meta as a public variable which could be used.

@gouthambs Thoughts?

Add GeoIP Support

Currently this only supports Freegeoip.net which is currently returning Bad Gateway. A second service should be added as an alternative/replacement in the event the other end GeoIP service goes down.

Adding support for mongo_engine.MongoEngineStorage

I'm volunteering to write, test, & document this expansion, but wanted to raise it as a issue first to make sure of no complications.

Essentially, it is not too different than the MongoPiggybackStorage variant already written. But it would be referenced the manner someone using the MongoEngine wrapper library would expect. Something like:

from flask.ext.track_usage import TrackUsage
from flask.ext.track_usage.storage.mongo_engine import MongoEngineStorage

t = TrackUsage(app, MongoEngineStorage())

The only reason MongoEngineStorage would be stored in .storage.mongo_engine rather than .storage.mongo would be to prevent a needless library dependency for those not using MongoEngine.

If the user were to want to change a characteristic of the storage they would do it with a MongoEngine class inheritance of the default. For example to change the name of the collection:

from flask.ext.track_usage import TrackUsage
from flask.ext.track_usage.storage.mongo_engine import MongoEngineStorage, UsageTracker

class MyTrackUsage(UsageTracker):
    meta = {'collection': 'websiteDUsage'}

t = TrackUsage(app, MongoEngineStorage(doc=MyTrackUsage))

I also have a second expansion idea, but I'm not sure it is best placed into this library. I'll post a second issue to discuss that.

Add Support for Other SQL databases

@ashcrow
This is a great package. Saved me a lot of time when I wanted a light usage tracker. I noticed that you don't support other storages such as SQLite of MySQL. I have added a Storage based on SQLAlchemy. If you are interested I can create a pull request.

Release 1.0.1

Release it.

Project still maintained?

Hi @ashcrow love this project! Is it still maintained? Do you still accept PRs?

SQLStorage is missing blueprint column

I forgot to add blueprint column to the SQLStorage. Will fix and issue a pull request for this issue.

pymongo no longer uses pymongo.Connection

It is replaced with pymongo.MongoClient.

Release 1.1.1

Release 1.1.1