jonparker / green-webhook

💚 💚 GreenWebhook 💚 💚 is a proxy or gateway between two systems that helps reduce your carbon footprint by dynamically routing or delaying traffic so that your workloads run when and where they will cause the lowest amount of carbon emissions.

💚 Welcome to Green Webhook! 💚

Green Webhook is a proxy or gateway between two systems that intelligently chooses which endpoint to send a webhook event to in order to location shift or time shift (coming soon) compute workloads to the endpoint with the lowest carbon intensity. It uses the Carbon Aware SDK to check the impact of an increased demand on the electricity grid for the relevant locations.

By deploying your workload to multiple regions and configuring each endpoint along with its location within a Green Webhook you can reduce the carbon emissions caused by your workload.

Currently Green Webhook supports only Azure regions but as the Carbon Aware SDK adds more support for other locations Green Webhook will be able to support them too.

Green Webhook is built on RedwoodJS and uses:

Create a webhook that runs code on your laptop via ngrok

1. Create a NextJS app by running npx create-next-app nextjs-blog --use-npm --example "https://github.com/vercel/next-learn/tree/master/basics/api-routes-starter"
2. Create pages/api/user.js with:

export default function handler(req, res) {
  res.status(200).json({ name: "John Doe" });
}

3. In a terminal run npm run dev
4. Install Ngrok
5. In a terminal run ngrok http 3000
6. Copy the URL output by Ngrok and add /api/user to the end
7. Log in to https://greenwebhook.netlify.app and create a new webhook
8. Paste the URL from step 6 into the destination
9. Click Save
10. Click on the Invocation URI link to trigger the webhook
11. Confirm that the response is 200 and contains JSON { name: "John Doe" }

Currently the Carbon Aware SDK only supports a limited list of locations but when more locations are supported then we will be able to allow you to put in the location where your computer or laptop is running in order to determine if it is the location with the lowest carbon intensity.

Getting started developing locally

Prerequisites

• Green Webhook is built with Redwood and requires Node.js (>=14.19.x <=16.x) and Yarn (>=1.15)
• Are you on Windows? For best results, follow our Windows development setup guide

Start by installing dependencies:

yarn install

Whilst that is running, create a file in the root folder called .env and copy the following into it:

# Used to encrypt/decrypt session cookies. Change this value and re-deploy to log out all users of your app at once.
# Run yarn redwood generate secret and copy the value below
SESSION_SECRET=YOUR_SESSION_SECRET

# SendInBlue secret for reset password emails to work (optional)
# Requires setting up a free account at https://www.sendinblue.com/
SEND_IN_BLUE_KEY=YOUR_SENDINBLUE_SECRET
SEND_IN_BLUE_EMAIL=YOUr_SENDINBLUE_EMAIL_ADDRESS

DEPLOY_URL=http://localhost:8910

# Create a free database via https://railway.app and replace YOUR_RAILWAY_DB_URL bellow with the connection URL
DATABASE_URL="YOUR_RAILWAY_DB_URL"
CARBON_AWARE_API_BASE_URI="https://carbon-aware-api.azurewebsites.net"

# Run yarn redwood generate secret and copy the value below
SCHEDULED_WEBHOOKS_API_KEY=YOUR_SCHEDULED_WEBHOOK_SECRET_KEY

Replace YOUR_SESSION_SECRET with the secret generated by running yarn rw generate secret.

Replace YOUR_SCHEDULED_WEBHOOK_SECRET_KEY with the secret generated by running yarn rw generate secret a second time.

Create a free database via https://railway.app and copy the connection URL and replace YOUR_RAILWAY_DB_URL with it

If you want to be able to send emails to reset passwords then you will need to create a SendInBlue account and create an API key and replace SEND_IN_BLUE_KEY with the API key.

Now when yarn install is finished you're ready to create the database tables in your Railway database.

yarn redwood prisma migrate dev

Once this finishes you should see tables appear in the database in Railway

Now you can run the app locally:

yarn rw dev

Your browser should automatically open to http://localhost:8910 where you'll see the Home page.

If you want to run scheduled webhooks you will need to periodically send a GET request to http://localhost:8910/.netlify/functions/schedule with the x-api-key header set to the YOUR_SCHEDULED_WEBHOOK_SECRET_KEY secret you created above.

Frontend first with Storybook

Don't know what your data models look like? That's more than ok — Redwood integrates Storybook so that you can work on design without worrying about data. Mockup, build, and verify your React components, even in complete isolation from the backend:

yarn rw storybook

green-webhook's Issues

Include http headers for request and response

Fix create new webhook button

Clicking on the create new green webhook button doesn't do anything.

Possibly there is something wrong in web/src/Routes.tsx but I haven't been able to work out what it is.

[Epic] Add support for latitude and longitude in location

This is a large piece of work because it requires adding support for latitude and longitude based support to the Carbon Aware API.

Create favicon from logo

[BUG] Viewing webhooks with new dark theme is broken

Destination endpoints are white on white background.

Remove the invocation uri from the frontend

Because we're using the id to generate the invocation URI we don't need to display the invocation URI field in the frontend.
Optionally the field could be removed from the database entirely but that's not as important as removing it from the frontend.

Update README with project brief

At the moment the README is just what was generated by RedwoodJS so we need to update it to include description of what the Green Webhook project is.

Implement external auth

Rather than use username and password auth stored in the DB an external provider should be used.
Alternatively a magic link in an email could be used for all sessions.

Write text project brief

Cover:

1. The problem we're solving
2. Our solution
3. How it can be taken from it's current state into a production application
4. Links to Github https://github.com/jonparker/green-webhook and application https://greenwebhook.netlify.app/
5. How are we using the Carbon Aware SDK?
6. What is the vision for the project and how it could help reduce carbon emissions.
7. Anything else?

Create Ngrok + Green Webhook tutorial

Add menu to forgot password page

Setup Gitbook

Kickoff meeting

Agree on a time.
Start an agenda

Password validation on signup

When a user creates an account or resets their password it should validate that the password is at least 8 characters has a strong complexity.

Host Carbon Aware SDK

The Carbon Aware SDK needs to be hosted somewhere that the Green Webhook can use it.
This will also require getting a Watttime API key.

Planning of user stories / features

Ideally this would be a group session but depending on our availability we might need to do this asynchronosly via chat.

1. OPTIONAL Setting up the authentication (currently using database) with RedwoodJS
2. DONE Setting up the deployment (choosing database hosting)
3. Work out how we can schedule code to run with RedwoodJS (we're hosting on Netlify so most likely Netlify scheduled functions would be the best way https://docs.netlify.com/functions/scheduled-functions/ )
4. DONE Setting up TailwindCSS
5. Creating a Logo
6. OPTIONAL Choosing a domain name (we already have a custom subdomain on Netlify)

Add example to Gitbook of using Azure and serverless framework

Add time shifting support

In order to support time shifting ( delaying calling the webhook endpoint ) we need to setup scheduled functions in RedwoodJS / Netlify https://docs.netlify.com/functions/scheduled-functions/

Supabase has support ( https://supabase.com/blog/postgres-as-a-cron-server ) but we're currently hosting the database on Railway which has support but requires a few more steps ( https://blog.railway.app/p/cron-jobs )

Include an optional estimated processing time field

Include an estimated processing time field in the webhook form which would be optional.

What is needed are two things,

1. a hidden hasEstimate boolean field that is passed along with the form data
2. a estimated Time in seconds (optional) field that accepts an integer (1 upto 86399)

The hasEstimate field has false by default and true if any input is provided in the estimated Time field.

You may also include a note saying

if blank, estimation will be done based on actual processing times

Remove null locations from location list

Those locations whose Longitude and Latitude are listed as null in carbon aware sdk's azure regions are also listed in avaiable in "Create New Webhook".

Selecting these locations gives 500 Internal Server Error

Add invocation history (logs)

Create new green webhook button is broken

It goes to this link https://greenwebhook.netlify.app/newWebhook which doesn't exist.

Create project logo

Jade:

This was an idea I found for the logo. Any thoughts on this?

Jon

It looks great. You should add it to the project if you want.

Start styling the UI/UX Frontend

** Jon **
I haven't thought much about styling but it would be nice to have a some green in the colour pallette.

Update title in browser

Change title to Green Webhook

Create a scheduled invocations page for each webhook

In the /webhooks route, where all webhooks are listed, create a show "schedules" button.

On the new page, list all the schedules available for that webhook and seaprate them into two switchable menus of "scheduled" and "completed" based on the isCompleted boolean present in each element of array

So,

query FindWebhookById($id: String!) {
    webhook: webhook(id: $id) {
      id
      include: {
        scheduledInvocations: true
      }
    }
  }

could do the trick

Endpoints array size is fixed to 2

Even if a second endpoint is not filled on the create webhook page, the endpoints array is always of fixed length 2 like

endpoints  [
  'https://--first-endpoint-url--/api/user|asiapacific',
   '|na'
]

This leads to execution always going to the else block below.

green-webhook/api/src/functions/webhook/webhook.ts

Lines 68 to 76 in 9653c44

	const endpoints = webhook.destinationEndpoints.split(',')
	let bestEndpoint = null
	if (webhook.maxDelaySeconds > 0 && endpoints.length === 1) {
	// Check Carbon Aware SDK for best time within maxDelaySeconds to call endpoint.
	} else {
	// Check Carbon Aware SDK for best location
	const locations = endpoints.map((e) => e.split('|')[1])

There can be multiple approaches

1. check for '|na' before or right here in the code and remove it from array
2. modify frontend to show only 1 endpoint form and a "+" button for users to add more endpoints

Add caching of Carbon Aware data

If we have many requests that require getting the same data from the Carbon Aware API we should cache the data for 5 or 10 minutes.

The data couuld be cached in the database or just in memory (which may be more difficult as the code is running in a serverless function).

Success confirmation feature

Add a feature that allows for the endpoint to call back to the webhook on some URL to notify that the job or workload has completed. This would enable more analytics of how much carbon emissions were saved by the Green Webhook moving the workload.

Note that this wouldn't give an actual Kg of CO2 measurement as it would take much more work to know how many Killowatts of power the workload consumed.