purdueelectricracing / fullsend

Automated email service for the Purdue Electric Racing team.

FullSend

FullSend is an automated email service that helps the Purdue Electric Racing team manage mass communication in and around our organization. This was built using Microsoft Graph integration with our Purdue emails, and is only accessible to students who are verified to be on the Purdue Electric Racing team.

Prerequisites

• Python version 3.5.2 or later.

Development environment

• Create a new virtualenv: python3 -m venv venv
• Activate: source venv/bin/activate
• Install prerequisistes pip install -r requirements.txt

Register the app

Head over to https://apps.dev.microsoft.com to quickly get a application ID and password. Click the Sign in link and sign in with either your Microsoft account (Outlook.com), or your work or school account (Office 365).

Once you're signed in, click the Add an app button. Enter FullSend for the name and click Create application. After the app is created, locate the Application Secrets section, and click the Generate New Password button. Copy the password now and save it to a safe place. Once you've copied the password, click Ok.

Locate the Platforms section, and click Add Platform. Choose Web, then enter http://localhost:8000/mail/gettoken/ under Redirect URIs.

NOTE: The values in Redirect URIs are case-sensitive, so be sure to match the case!

Click Save to complete the registration. Copy the Application Id and save it along with the password you copied earlier. We'll need those values soon.

Configure the sample

1. Open the .\FullSend\authhelper.py file.
2. Replace YOUR APP ID HERE with the Application Id from the registration you just created.
3. Replace YOUR APP PASSWORD HERE with the password you copied earlier.
4. Run migrations by entering python3 manage.py migrate from the command prompt.
5. Run the project by entering python3 manage.py runserver from the command prompt.

Services used

• Sentry.io - Used to monitor all errors
• UptimeRobot - Used to keep heroku dyno active and monitor server uptime.

Acknowledgements

This project was started using the Outlook REST APIs Python Tutorial.

fullsend's Issues

Create example preview

Next page should show the example email with the parts filled in. Also, show any errors (like mismatched fields). If approved, FULL SEND!

Use venv

https://medium.com/@jtpaasch/the-right-way-to-use-virtual-environments-1bc255a0cba7

Add CSV upload/parser

Broken down from #3 , need a CSV parser to list out all items parsed.

Adjust gists to not require hard coding

Any email template currently requires a subject in content map:
(api/googleservice.py)

subject_map = {
    'welcome': 'Thanks for filling out our interest form!',
    'acceptance': 'Welcome to Purdue Electric Racing!',
}

content_map = {
    'welcome': 'welcome.html',
    'acceptance': 'acceptance.html',
}

Both of these can be done in the gist itself.

Quick send tab

Send to one individual with text boxes for each parameter instead of uploading the CSV.

Newline in content doesn't create new line in email

When I do newline on content form, it's ignored in the email HTML.

Error in text processing

In case you don\u2019t know who we are, supposed to be In case you don't know who we are

Create form to send mail

Add a textbox that contains the email body and options to send individually or as BCC. Ideally, contains a list of parameters grabbed from the CSV and a list of parameters parsed in the email, make sure they line up.

Putting the same element in multiple places doesn't replace all instances

When I had two {{COMPANY}} elements, only the first one was replaced.

Configure app for Heroku deployment

Start using environment variables according to https://devcenter.heroku.com/articles/config-vars

Running app locally: https://devcenter.heroku.com/articles/heroku-local

Add ability to send mail

Change app permissions to create and send mail along with read.

Expose API to send mail

Google form will have a form submit hook that immediately sends them a response email, can use the quick email to format

Add automatic token renewal for /api

Once we authenticate for the API, we don't want to lose that authentication. Perform checks every 5 minutes (notify through Sentry if it's found to be down), and every xx minutes (45 minutes) use refresh_token to get the next token.

Create base app

Base requirement to use with Outlook. However, it may be necessary to use Outlook APIs directly if a feature if required that isn't available on Graph endpoints.

Redirect URI does not match server Redirect URI

Configuration in server doesn't allow for redirection back to https://full-send.herokuapp.com

https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-configure-app-access-web-apis#add-redirect-uris-to-your-application

Change permissions in Azure to allow for redirection

Change SECRET_KEY

https://stackoverflow.com/questions/15170637/effects-of-changing-djangos-secret-key
https://stackoverflow.com/questions/4664724/distributing-django-projects-with-unique-secret-keys

Shouldn't change it once in production, but it's OK for now.

Modify CSRF to ignore for /api

https://stackoverflow.com/a/31136060/5615790

The user will authenticate with a key instead of a CSRF token, so ignore for the /api routes.

Add email configurations

Based on #20, the exposed API needs to know what mail to send. There should be a type in the api format that determines which title/body to use. This information should be uploaded to Google Drive (body as HTML), then queried.
Example:
API Query: type: formresponse
Google Drive (inside predefined folder): formresponse.html contains the body of the email to be sent

Create form to use during one-on-one

During one-on-one meeting, the lead should fill out a form that contains information about the person (name, email) - information that could be grabbed from interest form (use #21 to get all up-to-date info). Additionally, provide talking points and questions to ask the person.

The following must be collected: subteam recommendation, training recommendation. If the lead feels the student is capable enough where they don't require additional training (via the project), then don't include the part about training in the email.

Log all emails sent through this

https://docs.djangoproject.com/en/2.1/intro/tutorial02/

Add sentry.io

https://www.pythoncircle.com/post/30/how-to-upload-and-process-the-csv-file-in-django/

Server Error: Too many connections for "fuztyzcqipudmu"

Grabbed from Sentry: https://sentry.io/share/issue/546d505def0e4686ad03ddea18f27422/
fuztyzcqipudmu is the name of the user when getting the ServerAuthentication object from the database.

I think this issue is due to the fact that the scheduler has multiple jobs running at the same time, when only one is supposed to be running concurrently.

scheduler.add_job(refreshtoken, 'cron', minute='50', jitter=240)

Does not have max_instances set, therefore it is calling to update the token multiple times (at the same time).

See update log -- notice how upwards of 6 "update" events are sent in an hour (expected 1-2):
https://sentry.io/share/issue/695f8d83db8f405cba5484a87610a66e/

Make email sending asynchronous

Send email notification to authenticated user with a report of who it was sent to

Restrict users to be Purdue only

Users shouldn't be able to auth if they don't have a @purdue.edu email.

Add Google Sheets integration

Use in place of database so data is still viewable to people without using the website. Eventually can move to a server-hosted database, but for now this will be quicker. Authenticate as [email protected] with all the server settings in heroku config.

Add HTML upload for content

Have templates folder in bizzop folder that allows us to use templates

Thought about having templates with the tool but determined it would be better to just use HTML on google drive in case people don't want to use the tool.

"Default" response for anything related to callout

Recommend button for callout if the subject or contents has callout or call out in the contents/title.