liddiard / google-sheet-s3

Google Apps Script that publishes a Google Sheet to Amazon S3 as a JSON file. Auto-updates on edit & maintains data types. Creates an array of objects keyed by column header.

google-sheet-s3

A Google Apps Script that publishes a Google Sheet to Amazon S3 as a JSON file. Creates an array of objects keyed by column header, maintaining data types like numbers and booleans.

For example, it turns a spreadsheet like this:

Into an S3-hosted JSON file like this:

[
  {
    "headline": "Lorem ipsum dolor sit amet",
    "body": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Tristique senectus et netus et malesuada fames ac turpis egestas. Purus gravida quis blandit turpis cursus in. ",
    "author": "hliddiard",
    "post_time": "2022-12-28T23:44:55.000Z",
    "publish": true
  },
  {
    "headline": "Ut enim ad minim veniam",
    "body": "Eget lorem dolor sed viverra. Ipsum suspendisse ultrices gravida dictum fusce ut.",
    "author": "jdoe",
    "post_time": "2022-12-29T02:20:01.000Z",
    "publish": true
  },
  {
    "headline": "Duis aute irure dolor",
    "body": "Aliquam id diam maecenas ultricies mi eget mauris pharetra. Sit amet dictum sit amet justo donec enim diam vulputate. Massa ultricies mi quis hendrerit. Rutrum quisque non tellus orci.",
    "author": "jdoe",
    "post_time": "2022-12-29T17:10:22.000Z",
    "publish": true
  },
  {
    "headline": "Excepteur sint occaecat cupidatat non proident",
    "body": "Pellentesque habitant morbi tristique senectus et netus et. Est ultricies integer quis auctor elit sed vulputate. Donec et odio pellentesque diam volutpat commodo sed. Sapien et ligula ullamcorper malesuada proin. Accumsan sit amet nulla facilisi morbi tempus.",
    "author": "hliddiard",
    "post_time": "2022-12-29T19:45:05.000Z",
    "publish": false
  }
]

Get the add-on here on the Chrome Web Store.

2023 update: Google has unpublished the add-on for not being up-to-date with their latest developer requirements. I'm working on figuring out the best way to let others install/use it easily. In the meantime, you can use it by following the development setup instructions below.

Why?

Use case

"I want to display simple, structured, spreadsheet-like, publicly accessible data on a website (possibly with thousands of simultaneous visitors) that is easily updatable (possibly by multiple people at once) without the overhead and time of coding, deploying, and maintaining a database, API, and admin interface."

Examples

Staff directory list, restaurant menu items listing, sports team standings page, live blog.

Why not [alternative]?

• Doesn't require OAuth like the official Google Sheets API (no good for anonymous data viewing).
• Not using deprecated APIs like Tabletop.js that could suffer an untimely disappearance at the whims of Google.
• Doesn't require an intermediary web application like WSJ uses/used.
• Not an alternative service like Airtable that is powerful but costs 💰💰💰.
• Not slow at returning data like Google Apps Script Web Apps seem to be. (If you're okay with 2000ms response times, this solution is easier because it doesn't involve S3. S3 response times tend to be 10-20x faster.)

Setup

We need to get an Amazon Web Services (AWS) S3 bucket and your Google Sheet to talk to each other, so there are two parts to this setup: one for AWS, and one for the spreadsheet.

AWS setup

1. If you don't have one already, create an Amazon AWS account.
2. If you don't have one already, create an AWS S3 bucket.
3. If you don't have one already, create an IAM user that has write permission to your S3 bucket. You'll need these credentials later.
4. Add a bucket policy that enables public viewing of the published JSON. To enable, go to your S3 Management Console, click your bucket's name > Permissions tab > Bucket policy > enter your policy (sample to paste below) > click Save.
5. If you're going to be accessing the published JSON data from a web browser, you will also need to add a CORS policy to your S3 bucket that allows GET requests from whatever origin (domain name) you want to access your data from. To add a policy, go to your S3 Management Console, click your bucket's name > Permissions tab > CORS configuration > enter your policy (sample to paste below) > click Save.

Sample bucket policy

This configuration is what I imagine most people using this add-on will want. It allows public access of the data stored in your Google Sheet, but not public write access.

Replace the text "PUT-YOUR-BUCKET-NAME-HERE" with your bucket's name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AddPerm",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::PUT-YOUR-BUCKET-NAME-HERE/*"
        }
    ]
}

Sample CORS policy

This configuration will allow any web page on the internet to access your sheet's data. You may not want that. You can modify the line <AllowedOrigin>*</AllowedOrigin> and replace the asterisk with your own hostname; e.g. the domain name from which you will be making Ajax requests for the sheet's JSON.

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>Authorization</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Google Sheet setup

How to use the add-on after completing the above AWS setup.

1. Create or open an existing Google Sheet.
2. Format the sheet so that the first row contains the column headers you want your JSON objects to have as properties. Example:
3. Install and enable the add-on.
4. In the spreadsheet's menu, go to Add-ons > Publish to S3 > Configure...
5. Fill in the S3 bucket name, folder path within the bucket where you want data stored (leave blank for top level), and your AWS credentials that allow write access to the bucket.
6. Click "Save". The S3 URL of your JSON-ified spreadsheet will be shown.

Did I miss something in these instructions? Not working as expected? Feel free to file an issue.

Any time you want to update the published JSON, go to menu item Add-ons > Publish to S3 > Publish.

Usage notes

• The JSON file's filename is taken from the spreadsheet ID, so the spreadsheet can be safely renamed without breaking the URL.
• The add-on only looks at the sequentially first sheet tab (called "Sheet1" by default). It won't publish or respond to changes on other tabs.
• The add-on will ignore columns that don't have a value in the header (row 1) of the spreadsheet.
• The add-on will ignore empty rows, skipping over them to the next row with values.
• A blank cell in a row is represented in the JSON as null. So if you have a column that could have missing or optional values, be sure to handle the null value in your consuming code.

Development setup instructions

1. Create a new Google Apps Script with files whose names and content matches the ones in this repo (minus this readme).
2. In the menu bar, click Deploy > Test Deployments.
3. On the left sidebar, Select type > Editor Add-on.
4. Under Editor Add-on, press Create new test.
5. Version: Latest code, Config: Installed and enabled, Test document: (select a spreadsheet to use)
6. Press Done.
7. Select the saved test and press Execute.

Developement links for version published to Chrome Web Store

These links are not publicly accessible – please do not request access. They're only for my personal reference to develop the script. If you'd like to develop on this script yourself, follow the instructions above to set up a development environment.

• Sheet
• Apps Script

Credits

Logo attribution

• Cloud icon from onlinewebfonts.com
• Arrow by Alice Design from Noun Project
• Icon mask generated using Thomas Finch's iOS 7+ Icon Generator

google-sheet-s3's Issues

S3 JSON to Google Sheet

Can we get S3 json and put it in google sheet

support for multiple sheets on the same workbook

Each would have its own URL, probably taken from Sheet ID. Default sheet has ID 0.

Authorization mechanism not supported - "Please use AWS4-HMAC-SHA256"

Hi there,

I've gotten to the step where I enter the Bucket name, path (blank), AWS access key ID, and AWS secret key into the dialog box on the google sheet. However, when I click save, the page hangs with the button showing "Saving", and nothing occurs.

In the Chrome developer tools, I see the following error logged:

665147591-mae_html_user_bin_i18n_mae_html_user.js:42 

Uncaught Error: AWS Error - InvalidRequest: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
 at S3Request:184 (S3:4) 38692e5d-4673-4c76-8733-0d4f8f1a0aa0
 at S3:146 (S3:4) a1279393-c26a-481c-9557-516ab8f2ac22
 at publish (Code:71) (Publish sheet to S3:20) 744867ab-df1e-44d5-aa49-4744c8b83522
 at updateConfig (Code:100) (Publish sheet to S3:20) 744867ab-df1e-44d5-aa49-4744c8b83522

I think I've followed your instructions correctly and set up everything I need to on the S3 side. Any idea on how to get around this error?

Thanks!
Jeffrey

Add-on no longer supported?

Hi. When I click on the link for your "Publish to S3" add-on, it says the requested link is no longer on the server. Is this add-on no longer supported? Thanks