hivesolutions / appier Goto Github PK
View Code? Open in Web Editor NEWJoyful Python Web App development
Home Page: http://appier.hive.pt
License: Apache License 2.0
Joyful Python Web App development
Home Page: http://appier.hive.pt
License: Apache License 2.0
It should be really cool to create a migration infra-structure like the one present in Migratore.
The idea is to build an infra-structure that is agnostic from the underlying database manager used.
By using multicast UDP it would be possible to discover other services with the same INSTANCE values and start communicating with them for certain puposes.
Election system to determine the one that is considered the master.
For instance it should be possible to discover a configuration server that way, or for Ripe discover the proper composer instance in the current network and auto-configure it.
Add support for a Cron-like Scheduler
that can easily handle the schedule of simple tasks regularly.
This feature has the potential to greatly simplify the way regular job handling is done within the Appier context.
The solution should expose an interface similar to:
def schedule(self, task: function, cron: string):
pass
It would be desirable to be able to send pre-processed messages using this method, instead of only templates, however the method has no support for that.
The get_address method needs to parse the X-Forwarded-For header and return only the first one.
It also needs to check if other headers are used, at least x-real-ip and x-client-ip.
This is critical to run applications behind proxies.
It should be possible by using code to gather the complete set of ACL tokens that are registered for the currently running application. This way it would be possible to list the complete set of possible permission for the running application context.
Should be possible to use appier.get_tokens()
to retrieve the complete set of ACL tokens and action methods (from controllers) associated with them.
Currently the _validate()
method uses the validation methods in validate()
or validate_new()
, depending on whether the object is new or not. It's useful in some contexts to be able to call the _validate()
method an be able to tell it which validation methods it should execute, one or the other, without relying on the object being validated.
Using a pool of threads and the Future
based abstraction it should be possible to adapt the current data layer access to an async fashion.
PyMongo with callback support.
https://motor.readthedocs.io/en/latest/examples/callbacks-and-coroutines.html#with-coroutines
While using Appier it's important to keep in mind that many features of it use the concept of global variable to simplify the usage process.
These same global variables may created weird runtime issues while running the App, mainly while running multiple Apps in the same process.
To avoid some of thoses issues a per-thread association should be created between an App and the thread where it's container (web server) is running.
The start()
and stop()
method of an app are called "inside" the web server container and should be used for this kind of registration.
After conducting some empirical testing, Appier is underperforming regarding async call handling.
Running the same App through FastAPI handles at least 10x-100x more requests than Appier.
Try to find a fix for the issues affecting Appier's async call handling.
Verify: there's some timeout kind of waiting going on, it is something to look into.
It's important to have a mechanism that allows control of the number of requests per time unit. This will provide a way to control an unintentional (or intentional) DOS.
Create a decorator appier.requests
that receives the number of requests per minute allowed for the action method in the controller.
The implementation should take inspiration from PreflightPart
.
Calling new()
on a model with relations causes an error like the following:
AttributeError
500 - 'module' object has no attribute 'Wishlist'
Traceback (most recent call last):
File "/repo.extra/appier/src/appier/base.py", line 803, in application_l
result = self.handle()
File "/repo.extra/appier/src/appier/base.py", line 931, in handle
else: result = self.route()
File "/repo.extra/appier/src/appier/base.py", line 1159, in route
else: return_v = method_i(*args, **kwargs)
File "/repo.extra/myswear/src/myswear/controllers/web/base.py", line 104, in do_signup
user = sadapters.models.User.new()
File "/repo.extra/appier/src/appier/model.py", line 354, in new
if fill: model = cls.fill(model)
File "/repo.extra/appier/src/appier/model.py", line 970, in fill
default = _type._default() if hasattr(_type, "_default") else default
File "/repo.extra/appier/src/appier/typesf.py", line 424, in _default
return cls(None)
File "/repo.extra/appier/src/appier/typesf.py", line 373, in __init__
self.__start__()
File "/repo.extra/appier/src/appier/typesf.py", line 417, in __start__
if is_reference: self._target = self.__class__._target()
File "/repo.extra/appier/src/appier/typesf.py", line 428, in _target
if is_reference: return getattr(common.base().APP.models_i, target)
AttributeError: 'module' object has no attribute 'Wishlist'
When calling map()
on a model, the attributes that are defined in the model are set in the map, but the undefined ones aren't. It would be desirable to be able to include all attributes by passing a special flag to the method.
The below sample app starts, runs and answers requests, but some timeout (about 30%):
#!/usr/bin/python
# -*- coding: utf-8 -*-
import appier
import appier_extras
class App(appier.WebApp):
def __init__(self, *args, **kwargs):
appier.WebApp.__init__(
self, name="app", parts=(appier_extras.AdminPart,), *args, **kwargs
)
self._load_scales()
@appier.route("/bundles", "GET")
def get_bundles(self):
return self.bundles
def _load_scales(self):
url = "https://api.github.com/"
result = appier.get(url)
locales = {"en_us": result}
for locale, bundle in appier.legacy.iteritems(locales):
self._register_bundle(bundle, locale, context="scales")
if __name__ == "__main__":
app = App()
app.serve()
However, the following has no issues:
#!/usr/bin/python
# -*- coding: utf-8 -*-
import appier
import appier_extras
class App(appier.WebApp):
def __init__(self, *args, **kwargs):
appier.WebApp.__init__(
self, name="app", parts=(appier_extras.AdminPart,), *args, **kwargs
)
self._load_scales()
@appier.route("/bundles", "GET")
def get_bundles(self):
return self.bundles
def _load_scales(self):
locales = {"en_us": {"current_user_url": "https://api.github.com/user"}}
for locale, bundle in appier.legacy.iteritems(locales):
self._register_bundle(bundle, locale, context="scales")
if __name__ == "__main__":
app = App()
app.serve()
In my local development machine, both work. In my docker deployed app, the first fails and the second works.
We had an hypothesis where the HTTP call in the init method was invalid since the event loop was still warming up. We replaced the appier.get call for a requests.get (python requests lib) and it worked:
#!/usr/bin/python
# -*- coding: utf-8 -*-
import appier
import appier_extras
class App(appier.WebApp):
def __init__(self, *args, **kwargs):
appier.WebApp.__init__(
self, name="app", parts=(appier_extras.AdminPart,), *args, **kwargs
)
self._load_scales()
@appier.route("/bundles", "GET")
def get_bundles(self):
return self.bundles
def _load_scales(self):
import requests
url = "https://api.github.com/"
result = requests.get(url)
result = result.json()
locales = {"en_us": result}
for locale, bundle in appier.legacy.iteritems(locales):
self._register_bundle(bundle, locale, context="scales")
if __name__ == "__main__":
app = App()
app.serve()
This is an issue whether the HTTP remote call is made in the init method or the start method. I would assume Appier's thread pool manager is still warming up and such an early HTTP call is corrupting it in some way.
Appier has an email client but it's currently referenced anywhere in the documentation. A simple example on how to use the client should be added to the docs.
It should be possible for someone to print out the current date or its individual parts (year, month, day, hours, minutes, seconds).
Making the datetime module acessible in the template should solve the problem.
Currently it's possible to store an integer on a string based field using the Appier data model storage engine. This is a serious vulnerability that may lead to corruption of the database.
Ensure proper validation of the data types on storage/persist operations.
Sometimes it's important to have a proper abstraction for a working unit inside an abstraction system.
Having per example a DownloadTask
that allows the downloading of an HTTP based asset and that properly notifies any listening handler should be the aim of this operation.
Create the Task
class abstraction and then use the websocket infra-structure to make Appier Admin control all of these tasks.
There should be some co-relation between this Task
class and the Future
class.
Appier routes like this:
@appier.route("/api/orders/<int:number>/attachments/<str:attachment_name>", "GET", json = True)
work as expected and match with an URL such as:
https://URL/api/9999/attachments/docname.pdf
However, they do not match the following two:
https://URL/api/9999/attachments/doc%20name.pdf
https://URL/api/9999/attachments/docname(1).pdf
It seems spaces and parenthesis are not supported. The resources are encoded by the likes of a method like JavaScripts' encodeURIComponent
and are valid URL syntax, so they should be supported.
SyntaxWarning: invalid escape sequence
Performing system checks...
System check identified no issues (0 silenced).
March 19, 2024 - 17:10:56
Django version 5.0.2
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
/usr/local/lib/python3.12/site-packages/appier/base.py:197: SyntaxWarning: invalid escape sequence '\('
REPLACE_REGEX = re.compile("(?<!\(\?P)\<((\w+)(\([\"'].*?[\"']\))?:)?(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:203: SyntaxWarning: invalid escape sequence '\<'
INT_REGEX = re.compile("\<int:(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:207: SyntaxWarning: invalid escape sequence '\<'
REGEX_REGEX = re.compile("\<regex\([\"'](.*?)[\"']\):(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:220: SyntaxWarning: invalid escape sequence '\('
CSS_ABS_REGEX = re.compile(b"url\((?!(http:\/\/|https:\/\/|\/\/|\/))([^\)]+)\)")
/usr/local/lib/python3.12/site-packages/appier/validation.py:40: SyntaxWarning: invalid escape sequence '\:'
SIMPLE_REGEX_VALUE = "^[\:\.\s\w-]+$"
/usr/local/lib/python3.12/site-packages/appier/validation.py:45: SyntaxWarning: invalid escape sequence '\w'
EMAIL_REGEX_VALUE = "^[\w\d\._%+-]+@[\w\d\.\-]+$"
/usr/local/lib/python3.12/site-packages/appier/validation.py:50: SyntaxWarning: invalid escape sequence '\w'
"^\w+\:\/\/([^@]+\:[^@]+@)?[^\:\/\?#]+(\:\d+)?(\/[^\?#]+)*\/?(\?[^#]*)?(#.*)?$"
Watching for file changes with StatReloader
/usr/local/lib/python3.12/site-packages/appier/util.py:1101: SyntaxWarning: invalid escape sequence '\`'
"""
/usr/local/lib/python3.12/site-packages/appier/base.py:197: SyntaxWarning: invalid escape sequence '\('
REPLACE_REGEX = re.compile("(?<!\(\?P)\<((\w+)(\([\"'].*?[\"']\))?:)?(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:203: SyntaxWarning: invalid escape sequence '\<'
INT_REGEX = re.compile("\<int:(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:207: SyntaxWarning: invalid escape sequence '\<'
REGEX_REGEX = re.compile("\<regex\([\"'](.*?)[\"']\):(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:220: SyntaxWarning: invalid escape sequence '\('
CSS_ABS_REGEX = re.compile(b"url\((?!(http:\/\/|https:\/\/|\/\/|\/))([^\)]+)\)")
/usr/local/lib/python3.12/site-packages/appier/validation.py:40: SyntaxWarning: invalid escape sequence '\:'
SIMPLE_REGEX_VALUE = "^[\:\.\s\w-]+$"
/usr/local/lib/python3.12/site-packages/appier/validation.py:45: SyntaxWarning: invalid escape sequence '\w'
EMAIL_REGEX_VALUE = "^[\w\d\._%+-]+@[\w\d\.\-]+$"
/usr/local/lib/python3.12/site-packages/appier/validation.py:50: SyntaxWarning: invalid escape sequence '\w'
"^\w+\:\/\/([^@]+\:[^@]+@)?[^\:\/\?#]+(\:\d+)?(\/[^\?#]+)*\/?(\?[^#]*)?(#.*)?$"
[19/Mar/2024 17:11:00] "GET /api/rgp/list HTTP/1.1" 200 820
/usr/local/lib/python3.12/site-packages/appier/util.py:1101: SyntaxWarning: invalid escape sequence '\`'
"""
/usr/local/lib/python3.12/site-packages/appier/base.py:197: SyntaxWarning: invalid escape sequence '\('
REPLACE_REGEX = re.compile("(?<!\(\?P)\<((\w+)(\([\"'].*?[\"']\))?:)?(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:203: SyntaxWarning: invalid escape sequence '\<'
INT_REGEX = re.compile("\<int:(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:207: SyntaxWarning: invalid escape sequence '\<'
REGEX_REGEX = re.compile("\<regex\([\"'](.*?)[\"']\):(\w+)\>")
/usr/local/lib/python3.12/site-packages/appier/base.py:220: SyntaxWarning: invalid escape sequence '\('
CSS_ABS_REGEX = re.compile(b"url\((?!(http:\/\/|https:\/\/|\/\/|\/))([^\)]+)\)")
/usr/local/lib/python3.12/site-packages/appier/validation.py:40: SyntaxWarning: invalid escape sequence '\:'
SIMPLE_REGEX_VALUE = "^[\:\.\s\w-]+$"
/usr/local/lib/python3.12/site-packages/appier/validation.py:45: SyntaxWarning: invalid escape sequence '\w'
EMAIL_REGEX_VALUE = "^[\w\d\._%+-]+@[\w\d\.\-]+$"
/usr/local/lib/python3.12/site-packages/appier/validation.py:50: SyntaxWarning: invalid escape sequence '\w'
"^\w+\:\/\/([^@]+\:[^@]+@)?[^\:\/\?#]+(\:\d+)?(\/[^\?#]+)*\/?(\?[^#]*)?(#.*)?$"
The regex errors come from the use of "invalid escape sequences" in regex patterns defined within the appier library. In Python 3.6 and later, sequences like (, <, and : in regular expressions should be either double escaped (e.g., \() or, preferably, raw string literals should be used by prefixing the string with r, to avoid these warnings. This is because such sequences are not valid Python string escape sequences, and starting from Python 3.6, a SyntaxWarning is raised for these.
Here's an example to illustrate the adjustment:
Original code in appier that might be causing warnings:
REPLACE_REGEX = re.compile("(?<!\(\?P)\<((\w+)(\([\"'].*?[\"']\))?:)?(\w+)\>")
Adjusted code using a raw string literal:
REPLACE_REGEX = re.compile(r"(?<!\(\?P)\<((\w+)(\([\"'].*?[\"']\))?:)?(\w+)\>")
Appier has support for converting a model to a dictionary, however, it has no support for recursively doing so for the model's relations.
Here's a sample solution (checking for relations is not correct):
def map_full(self):
cls = self.__class__
map = self.map()
for name, value in appier.legacy.items(map):
_definition = cls.definition_n(name)
_type = _definition.get("type", None)
if not _type: continue
type_s = str(_type)
if type_s == "<class 'appier.typesf._Reference'>":
value = getattr(self, name)
map[name] = value.map_full()
elif type_s == "<class 'appier.typesf._References'>":
value = getattr(self, name)
map[name] = [_value.map_full() for _value in value]
return map
There seems to be no friendly way of extending an Appier session. In some contexts, it's desirable to be able to extend the session indefinitely.
The expiration date is being set when the session object is being created. This date can be changed only by accessing the internal property directly. Probably there should be an refresh()
method to easily refresh the session's expiration date (current date + default timeout).
Also, in RedisSession
values are being stored with SETEX
, therefore they are deleted by Redis when the timeout is reached. To extend the session in RedisSession
, the value's expiration date must be extended using the EXPIRE
command to provide a new timeout.
Sometimes it's relevant to add a remote log manager (rsyslog) to the main App instance from Appier.
APPIER_RSYSLOG=log.hive:514
or RSYSLOG=log.hive:514
https://docs.python.org/2/library/logging.handlers.html#sysloghandler
To be able to efficiently debug an Appier application it's very important to be able to have it reloaded at a development time. Effectively avoiding the need to restart the application & debugger.
Most of the implementation requires the checking of changes in the source files.
The overall algorithm can be defined as:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.