aerendir / bundle-aws-ses-monitor Goto Github PK

Hey,

I have been getting few exceptions about duplicate entry in email statuses table. Any ideas?

An exception occurred while executing 'INSERT INTO aws_ses_monitor_email_statuses (hard_bounces_count, soft_bounces_count, last_bounce_type, last_time_bounced, complaints_count, last_time_complained, deliveries_count, last_time_delivered, email_address) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [0, 0, null, null, 0, null, 1, "2017-08-21 11:01:16", "[email protected]"]: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "aws_ses_monitor_email_statuses_pkey" DETALHE: Key (email_address)=([email protected]) already exists.

I am running Sf 2.8 with "serendipity_hq/aws-ses-monitor-bundle": "^1.2"

As on Aerendir/bundle-stripe#35

As on

• Aerendir/bundle-commands-queues#54
• Aerendir/bundle-stripe#38

Goal: avoid to subscribe, for example, coommercio.com from trustback.me.

In development only email like [email protected] can be subscribed. Maybe can be useful to make this email configurable (dev.test_identity).

This requires #82

Todos:

• Configure on SES Identities configured in bundle:
  • Create the identity if doesn't already exist (SesClient::VerifyEmailAddress() and SesClient::VerifyDomainIdentity());
• Check the topic is not already subscribed
• Topic is subscribed to identities
• App is subscribed to topic
• Based on configuration, make possible to enable or disable the inclusion of original headers in the SNS notification (SetIdentityHeadersInNotificationsEnabled);
• Based on configuration, make possible to enable or disable the email feedback forwarding (SetIdentityFeedbackForwardingEnabled);
• Check differences for dev env and prod env: on dev should not be possible to subscribe to domain identities, but only to the email identity [email protected] (that has to be verified and work as expected to make possible to test the real sending of emails through SES in development). The test email can be configured: if changed, the debug command should highlight that the test email doesn't have the test mailbox.

This command will substitute the exsisting ones:

• aws:ses:monitor:setup:bounces-topic
• aws:ses:monitor:setup:complaints-topic
• aws:ses:monitor:setup:deliveries-topic

In the documentation, the examples of configuration should now use ENVs as this is the new way of setting parameters.

As on Aerendir/bundle-stripe#23

The enabled param is not sufficiently clear: rename it to track to make clear its purpose is to enable tracking also of the delivered emails.

The same for complaints and bounces.

https://github.com/symfony/recipes-contrib

• Create a config file in config/packages/shq_aws_ses_monitor.yaml with this content:

# Aws Ses Monitor
shq_aws_ses_monitor:
    aws_config:
        credentials_service_name: 'Aws\Credentials\Credentials'
        region: '%env(AWS_REGION)%'
    bounces:
        topic:
            name: '%env(AWS_SES_TOPIC_BOUNCES)%'
            endpoint:
                protocol: '%env(APP_SCHEME)%'
                host: '%env(APP_HOST)%'
    complaints:
        topic:
            name: '%env(AWS_SES_TOPIC_COMPLAINTS)%'
            endpoint:
                protocol: '%env(APP_SCHEME)%'
                host: '%env(APP_HOST)%'
    deliveries:
        topic:
            name: '%env(AWS_SES_TOPIC_DELIVERIES)%'
            endpoint:
                protocol: '%env(APP_SCHEME)%'
                host: '%env(APP_HOST)%'

• Create a config file in config/packages/dev/shq_aws_ses_monitor.yaml with this content:

# Aws Ses Monitor
shq_aws_ses_monitor:
    bounces:
        topic:
            endpoint:
                protocol: '%env(NGROK_APP_SCHEME)%'
                host: '%env(NGROK_APP_HOST)%'
    complaints:
        topic:
            endpoint:
                protocol: '%env(NGROK_APP_SCHEME)%'
                host: '%env(NGROK_APP_HOST)%'
    deliveries:
        topic:
            endpoint:
                protocol: '%env(NGROK_APP_SCHEME)%'
                host: '%env(NGROK_APP_HOST)%'

• Add those variables to .env file:
  • NGROK_APP_SCHEME
  • NGROK_APP_HOST
  • AWS_SES_VERSION
  • AWS_SES_TOPIC_BOUNCES
  • AWS_SES_TOPIC_COMPLAINTS
  • AWS_SES_TOPIC_DELIVERIES

Other variables used, but not only by AwsSesMonitorBundle
- AWS_SNS_VERSION
- AWS_ACCESS_KEY_ID
- AWS_ACCESS_KEY_SECRET
- AWS_REGION

As on

• Aerendir/bundle-stripe#20
• Aerendir/bundle-commands-queues#39

Make possible to see relevant information directly from the app front-end, using SonataAdmin.

As on

• Aerendir/bundle-stripe#32
• Aerendir/bundle-commands-queues#49

Goal: avoid to subscribe the same topic from development and production.

As on

• Aerendir/bundle-stripe#34

• Aerendir/bundle-commands-queues#51

• Remove badges from README.md

• Remove connection with CodeShip (on codeship and in GitHub)

Configure Travis-ci to test with both versions of Symfony and PHP.

Implement also other suggestions from best practices: Continuous integration.

As on

• Aerendir/bundle-commands-queues#44
• Aerendir/bundle-stripe#24

Rename

• shq_aws_ses_monitor.bounces.topic.endpoint.protocol to shq_aws_ses_monitor.bounces.topic.endpoint.scheme
• shq_aws_ses_monitor.complaints.topic.endpoint.protocol to shq_aws_ses_monitor.complaints.topic.endpoint.scheme
• shq_aws_ses_monitor.deliveries.topic.endpoint.protocol to shq_aws_ses_monitor.deliveries.topic.endpoint.scheme

See also #41.

Events that will be dispatched:

• Recipient filtered out (before sending);
• Recipient filtered out (after sending);
• Bounce notification from AWS SES;
• Complaint notification from AWS SES;
• Delivery notification from AWS SES;

Requested in #31

As on

• Aerendir/bundle-stripe#30
• Aerendir/bundle-commands-queues#47

See also #40.

As on Aerendir/bundle-commands-queues#46

It is not clear in the documentation the purpose of the parameter.

As on

• Aerendir/bundle-commands-queues#48
• Aerendir/bundle-stripe#31

As on

• Aerendir/bundle-commands-queues#50
• Aerendir/bundle-stripe#33

• https://stripe.com/docs/webhooks#best-practices
• http://stackoverflow.com/questions/35219537/symfony-run-code-after-response-was-sent/35219628#35219628

As on Aerendir/bundle-stripe#4

As on

• Aerendir/bundle-commands-queues#56
• Aerendir/bundle-stripe#41

The command has to check that all is configured well.

This requires #82

Checks to do:

• Account

  • The account is enabled for sending emails;
  • There still is quota available;

• Identities: list all identities and for each one list:

  • Is Identity verified?
  • Is DKIM enabled?
  • Is DKIM verified?

This requires phpstan/phpstan#1203 to be solved.

Then reactivate also the tests oh Phan.

[ ] Add query string param that is used as password;
[ ] Check if it is possible to verify the request really comes from Amazon.

As on Aerendir/bundle-stripe#2

https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html

The command is meant to unsubscribe a topic from a notification type.

https://docs.aws.amazon.com/ses/latest/DeveloperGuide/manage-sending-limits.html?icmpid=docs_ses_console

Currently the bundle requires a configuration like this:

# Aws Ses Monitor
shq_aws_ses_monitor:
    aws_config:
        credentials_service_name: 'Aws\Credentials\Credentials'
        region: '%env(AWS_REGION)%'
    bounces:
        topic:
            name: '%env(AWS_SES_TOPIC_BOUNCES)%'
            endpoint:
                protocol: '%env(APP_SCHEME)%'
                host: '%env(APP_HOST)%'
    complaints:
        topic:
            name: '%env(AWS_SES_TOPIC_COMPLAINTS)%'
            endpoint:
                protocol: '%env(APP_SCHEME)%'
                host: '%env(APP_HOST)%'
    deliveries:
        topic:
            name: '%env(AWS_SES_TOPIC_DELIVERIES)%'
            endpoint:
                protocol: '%env(APP_SCHEME)%'
                host: '%env(APP_HOST)%'

This is verbose and not useful.

Maybe a configuration like this is better:

# Aws Ses Monitor
shq_aws_ses_monitor:
    aws_config:
        credentials_service_name: 'Aws\Credentials\Credentials'
        region: '%env(AWS_REGION)%'
    topics:
        endpoint:
            protocol: '%env(APP_SCHEME)%'
            host: '%env(APP_HOST)%'
    bounces:
        # Remove
        topic:
            name: '%env(AWS_SES_TOPIC_BOUNCES)%'
    complaints:
        # Remove
        topic:
            name: '%env(AWS_SES_TOPIC_COMPLAINTS)%'
    deliveries:
        # Remove
        topic:
            name: '%env(AWS_SES_TOPIC_DELIVERIES)%'
            endpoint:
                protocol: '%env(APP__OTHER_SCHEME)%'
                host: '%env(APP_OTHER_HOST)%'

This way is the configuration is way more simple.

This will allow for more precise checks about subscriptions

As suggested by Best practices: Tests.

As on

• Aerendir/bundle-stripe#37
• Aerendir/bundle-commands-queues#53

https://docs.aws.amazon.com/sns/latest/dg/json-formats.html#http-header

x-amz-sns-message-type = UnsubscribeConfirmation

Relevant code in HandlerFactory::buildHandler().

As recommended by best practices: Directory structure.

Instead of requiring them to be added to the Doctrine's configuration.

See also #44.

Two issues when working with a custom entity manager:

In: aws-ses-monitor-bundle/Command/SnsSetupCommandAbstract.php

line 90 needs to be changed from:
$this->getContainer()->get('doctrine.orm.default_entity_manager')->flush();

to:
$this->getContainer()->get('aws_ses_monitor.entity_manager')->flush();

and line 181 from:
$this->getContainer()->get('doctrine.orm.default_entity_manager')->persist($topic);

to:
$this->getContainer()->get('aws_ses_monitor.entity_manager')->persist($topic);

Thanks!

As suggested by best practices: Documentation.

As on

• Aerendir/bundle-commands-queues#52
• Aerendir/bundle-stripe#36

Hey,

It would be nice to dispatch some events about bounces, complaints so that we can subscribe to make extra treatments in our app.

Regards

http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html#delivery-object

The AWS SNS has a MailObject part.

Save it into the database.

See: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notification-contents.html#mail-object

The corresponding identity has to be already verified or can be also not still verified?

This way is possible to call the command on each deploy to test if all works well.

Think better at this: the main goal is to be sure the integration with AWS SES works as expected each time the app is deployed.

For example, check the app can connect to AWS.

Make also possible to automatically subscribe all not already subscribed identities.

See also #42.

As recommended by best practices: Service Format: YAML

As on

• Aerendir/bundle-commands-queues#55
• Aerendir/bundle-stripe#39