kinto / kinto-emailer Goto Github PK
View Code? Open in Web Editor NEWPlugin to manage email notifications on records modification in a collection.
License: Other
Plugin to manage email notifications on records modification in a collection.
License: Other
By default collection_id
would be *
But in order to exclude a particular collection, we could specify ^(my-cid)
Dependabot couldn't authenticate with https://pypi.python.org/simple/.
You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.
Hi, i'm trying to setup kinto-emailer for new records notifications. I've added it to KINTO_INCLUDES
and created MAIL_DEFAULT_SENDER
and MAIL_HOST
as environment variables, but as soon as some notification is triggered i get this error message :
Unable to notify
Traceback (most recent call last):
File "/app/kinto/core/events.py", line 181, in _notify_resource_events_after
request.registry.notify(event)
File "/usr/local/lib/python3.7/site-packages/pyramid/registry.py", line 109, in notify
[_ for _ in self.subscribers(events, None)]
File "/usr/local/lib/python3.7/site-packages/zope/interface/registry.py", line 448, in subscribers
return self.adapters.subscribers(objects, provided)
File "/usr/local/lib/python3.7/site-packages/zope/interface/adapter.py", line 619, in subscribers
subscription(*objects)
File "/usr/local/lib/python3.7/site-packages/pyramid/config/adapters.py", line 129, in subscriber_wrapper
return derived_subscriber(*arg)
File "/usr/local/lib/python3.7/site-packages/pyramid/config/adapters.py", line 101, in derived_subscriber
return subscriber(arg[0])
File "/usr/local/lib/python3.7/site-packages/kinto_emailer/__init__.py", line 58, in send_notification
messages += get_messages(storage, _context)
File "/usr/local/lib/python3.7/site-packages/kinto_emailer/__init__.py", line 118, in get_messages
hooks = _get_emailer_hooks(storage, context)
File "/usr/local/lib/python3.7/site-packages/kinto_emailer/__init__.py", line 78, in _get_emailer_hooks
object_id=collection_id)
File "/app/kinto/core/decorators.py", line 70, in wrapper
return func(*args, **new_kwargs)
File "/app/kinto/core/storage/postgresql/__init__.py", line 339, in get
result = conn.execute(query, placeholders)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1291, in execute
return self._connection_for_bind(bind, close_with_result=True).execute(
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1147, in _connection_for_bind
engine, execution_options
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 458, in _connection_for_bind
self.session.dispatch.after_begin(self.session, self, conn)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/event/attr.py", line 261, in __call__
fn(*args, **kw)
File "/usr/local/lib/python3.7/site-packages/zope/sqlalchemy/datamanager.py", line 269, in after_begin
session, self.initial_state, self.transaction_manager, self.keep_session
File "/usr/local/lib/python3.7/site-packages/zope/sqlalchemy/datamanager.py", line 234, in join_transaction
session, initial_state, transaction_manager, keep_session=keep_session
File "/usr/local/lib/python3.7/site-packages/zope/sqlalchemy/datamanager.py", line 89, in __init__
transaction_manager.get().join(self)
File "/usr/local/lib/python3.7/site-packages/transaction/_transaction.py", line 191, in join
Status.ACTIVE, Status.DOOMED, self.status))
ValueError: expected txn status 'Active' or 'Doomed', but it's 'Committed'
So looks like the notification is triggered but something is missing in my setup. Hard to tell what with this error though.
Any guidance on how to debug this ?
In our use case for blocklists, it would be better to be able to define the configuration on the staging
bucket.
The groups URI could contain placeholder like this:
"/buckets/source/groups/{collection_id}-reviewers"
"hooks": [{
"event": "kinto_signer.events.ReviewRequested",
"subject": "{user_id} requested review on {bucket_id}/{collection_id}.",
"template": "Review changes at {root_url}admin/#/buckets/{bucket_id}/collections/{collection_id}/records",
"recipients": ["/buckets/source/groups/{collection_id}-reviewers"]
}]
// Another example.
{
"kinto-emailer": {
"sender": "Form Builder <[email protected]>",
"collection.created": {
"template": "You've just created a new form. Here are the details: \n- {email.data.formURL}\n -{email.data.formAdminURL}"
"recipients": ['{email.recipients}']
},
"record.created": {
"template": ""
"recipients": ['{email.recipients}']
}
}
}
/buckets/{buck}/collections/{collection}/emails
contains recipients, can be accessed only by people with the write permission on the collection.
The current implementation is wrong. If serveral collecions or several records are impacted by a batch request, only one email is sent.
For example, it uses payload['collection_id']
instead of iterating on impacted_records
.
Related: Kinto/kinto#945
This might be irrelevant but if we need to plug events from other plugins I guess we should add the ability to configure new events that should triggers emails from a common registry rather than using a try except close in the kinto-email includeme.
Currently notifications only works for records, it would be nice to be able to send notifications when the collection metadata are updated.
with a POST /v1/buckets
File "/usr/local/lib/python3.5/site-packages/zope/interface/registry.py", line 442, in subscribers
return self.adapters.subscribers(objects, provided)
File "/usr/local/lib/python3.5/site-packages/zope/interface/adapter.py", line 598, in subscribers
subscription(*objects)
File "/usr/local/lib/python3.5/site-packages/pyramid/config/adapters.py", line 130, in subscriber_wrapper
return derived_subscriber(*arg)
File "/usr/local/lib/python3.5/site-packages/pyramid/config/adapters.py", line 103, in derived_subscriber
return subscriber(arg[0])
File "/usr/local/lib/python3.5/site-packages/kinto_emailer/__init__.py", line 140, in _validate_emailer_settings
bucket_uri = '/buckets/{bucket_id}'.format(**event.payload)
KeyError: 'bucket_id'","agent":"HTTPie/0.9.9","collection_id":"bucket","authn_type":"ldap","path":"/v1/buckets","errno":110},"Pid":8,"Timestamp":1498670780707000000,"EnvVersion":"2.0"}
Hi, i've setup the plugin, and notifications are triggered correctly; Now i'm trying to configure MAIL_HOST
and MAIL_DEFAULT_SENDER
but looks like this is not configurable via env variables, only from the config file which is not ideal in my case.
While pyramid docs says one can use env vars to override configuration, i cant get it to work
Any idea where i should look to make this possible ?
Using kinto-ldap or kinto-portier the userID contains the email address.
It would be great to be able to set a group as recipients of an email notification.
{
"kinto-emailer": {
"sender": "Kinto team <[email protected]>"
"record": {
"recipients": ['Security reviewers <[email protected]>'],
"create": {
"template": "Hi, a new record '{uri}' has just been created in the collection '{bucket_id}/{collection_id}'",
},
"delete": {
"template": "Hi, a record '{uri}' has just been deleted in the collection '{bucket_id}/{collection_id}'",
}
}
"collection": {
"recipients": ['Admins <[email protected]>'],
"template": "Hi, a collection '{uri}' was {action}d.",
}
}
}
Also with the coming Account API, it would be even better if people could opt-out of notifications or explicitly ask for the kind of notifications you want to get.
Currently, only AfterResourceChanged
event is tracked.
Because we plug sending notifications to AfterResourceChanged
, we are out of the request transaction.
Because we use get_mailer().send()
, and because pyramid-tm is enabled, pyramid-emailer is waiting for the transaction to be committed to send emails.
The solution would be to use send_immediately(...)
instead of send()
. Or call transaction.commit()
but this could have side effects so I would suggest the former.
kinto_emailer\/__init__.py\", line 140, in _validate_emailer_settings\n bucket_uri = '\/buckets\/{bucket_id}'.format(**event.payload)
KeyError: 'bucket_id'```
Refs https://sentry.prod.mozaws.net/operations/kinto-prod-1/issues/618242/
$ kinto-emailer-test config/kinto.ini [email protected]
Email sent successfully.
While we are using the AfterResourceChanged
event we should probably try to run the email sending in an async process.
Refs: Kinto/kinto#422
Currently the placeholders can only refer to event attributes
https://github.com/Kinto/kinto-emailer#template
We would need to be able to inject the instance project name too.
eg: "subject": "[{settings.project_name}] Review requested"
/ref https://github.com/mozilla-services/remote-settings-permissions/pull/261
We're seeing an error from send_notification/get_messages
Error: ValueError("expected txn status 'Active' or 'Doomed', but it's 'Committed'")
Message: Unable to notify
Stack trace:
File "/opt/.venv/lib/python3.12/site-packages/kinto/core/events.py", line 190, in _notify_resource_events_after
request.registry.notify(event)
File "/opt/.venv/lib/python3.12/site-packages/pyramid/registry.py", line 103, in notify
[_ for _ in self.subscribers(events, None)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/zope/interface/registry.py", line 446, in subscribers
return self.adapters.subscribers(objects, provided)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/zope/interface/adapter.py", line 896, in subscribers
subscription(*objects)
File "/opt/.venv/lib/python3.12/site-packages/pyramid/config/adapters.py", line 125, in subscriber_wrapper
return derived_subscriber(*arg)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/pyramid/config/adapters.py", line 97, in derived_subscriber
return subscriber(arg[0])
^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/kinto_emailer/__init__.py", line 62, in send_notification
messages += get_messages(storage, _context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/kinto_emailer/__init__.py", line 117, in get_messages
hooks = _get_emailer_hooks(storage, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/kinto_emailer/__init__.py", line 80, in _get_emailer_hooks
metadata = storage.get(
^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/statsd/client/timer.py", line 41, in _wrapped
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/kinto/core/decorators.py", line 70, in wrapper
return func(*args, **new_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/kinto/core/storage/postgresql/__init__.py", line 341, in get
result = conn.execute(sa.text(query), placeholders)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2306, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2181, in _execute_internal
conn = self._connection_for_bind(bind)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2050, in _connection_for_bind
return trans._connection_for_bind(engine, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 2, in _connection_for_bind
File "/opt/.venv/lib/python3.12/site-packages/sqlalchemy/orm/state_changes.py", line 139, in _go
ret_value = fn(self, *arg, **kw)
^^^^^^^^^^^^^^^^^^^^
File "/opt/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1202, in _connection_for_bind
self.session.dispatch.after_begin(self.session, self, conn)
File "/opt/.venv/lib/python3.12/site-packages/sqlalchemy/event/attr.py", line 378, in __call__
fn(*args, **kw)
File "/opt/.venv/lib/python3.12/site-packages/zope/sqlalchemy/datamanager.py", line 302, in after_begin
join_transaction(
File "/opt/.venv/lib/python3.12/site-packages/zope/sqlalchemy/datamanager.py", line 265, in join_transaction
DataManager(
File "/opt/.venv/lib/python3.12/site-packages/zope/sqlalchemy/datamanager.py", line 117, in __init__
transaction_manager.get().join(self)
File "/opt/.venv/lib/python3.12/site-packages/transaction/_transaction.py", line 191, in join
raise ValueError(
<class 'ValueError'>
ValueError("expected txn status 'Active' or 'Doomed', but it's 'Committed'")
This occurs with the kinto-dist docker image and with no mail configuration specified.
GMar 01 21:05:13 ip-172-31-23-160 docker[6965]: Setting up 'changes' listener
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: Traceback (most recent call last):
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "app.wsgi", line 23, in <module>
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: application = main(config.items('DEFAULT'), **dict(config.items('app:main')))
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/kinto/__init__.py", line 52, in main
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: default_settings=DEFAULT_SETTINGS)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/kinto/core/initialization.py", line 586, in initialize
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: config.include("kinto.core", route_prefix=api_version)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/pyramid/config/__init__.py", line 800, in include
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: c(configurator)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/kinto/core/__init__.py", line 173, in includeme
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: config.include(app)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/pyramid/config/__init__.py", line 800, in include
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: c(configurator)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/kinto_emailer/__init__.py", line 179, in includeme
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: config.include('pyramid_mailer' + ('.debug' if debug else ''))
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/pyramid/config/__init__.py", line 800, in include
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: c(configurator)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/pyramid_mailer/debug.py", line 8, in includeme
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: mailer = DebugMailer(path)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/site-packages/pyramid_mailer/mailer.py", line 40, in __init__
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: makedirs(top_level_directory)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: File "/usr/local/lib/python3.5/os.py", line 241, in makedirs
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: mkdir(name, mode)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: PermissionError: [Errno 13] Permission denied: '/app/mail'
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: unable to load app 0 (mountpoint='') (callable not found or import error)
Mar 01 21:05:14 ip-172-31-23-160 docker[6965]: *** no app loaded. going in full dynamic mode ***
We may want to send two different emails for the same event. The current approach with the keys do not allow it.
I think also that we could use attributes to filter events, like this:
[
{
"resource_name": "record",
"action": "create",
"collection_id": "certificates",
"template": "Hi, a new record '{uri}' has just been created in the collection '{bucket_id}/{collection_id}'",
"recipients": ['Security reviewers <[email protected]>'],
"sender": "Kinto team <[email protected]>"
},
{
"event": "kinto_signer.events.ReviewRequested",
"template": "Hi, a review is pending your approval on the collection '{bucket_id}/{collection_id}'",
"recipients": ['Security reviewers <[email protected]>'],
"sender": "Kinto team <[email protected]>"
}
]
File "/home/mathieu/Code/Mozilla/kinto-emailer/.venv/local/lib/python2.7/site-packages/pyramid/config/adapters.py", line 103, in derived_subscriber
return subscriber(arg[0])
File "/home/mathieu/Code/Mozilla/kinto-emailer/kinto_emailer/__init__.py", line 24, in send_notification
messages = get_messages(collection_record, payload)
File "/home/mathieu/Code/Mozilla/kinto-emailer/kinto_emailer/__init__.py", line 53, in get_messages
msg = hook['template'].format(**payload)
KeyError: u'userid' lang=None uid=5eba31d5d5a47ea957928b1d38bfdb6d394d711796de85410bc9fbef2b2cec8d
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.