victorcouste / demo-trigger-dataprep-job-from-gcs

Assets for the demonstration of the blog post "How to Automate a Cloud Dataprep Pipeline When a File Arrives"

Trigger Cloud Dataprep jobs from GCS

Assets for demonstration of the blog post "How to Automate a Cloud Dataprep Pipeline When a File Arrives"(https://www.trifacta.com/blog/automate-cloud-dataprep-pipeline/)

• flow_dataprep_demo.zip : Dataprep Flow package example to import from Dataprep UI.

• Customers.csv : Customers CSV file used as one of inputs of the flow example and to be stored in the Google Cloud Storage you want to monitor.

• Occupation Mapping Google Sheet : Google Sheet used as one of inputs of the flow example for the lookup with the Customers csv dataset.

• cloud-function.py : Background Python function to trigger a Dataprep job when a file is created in a Google Cloud Storage bucket folder. Dataprep job started with REST API call and new file as parameter.

Google Cloud Functions https://cloud.google.com/functions

Google Cloud Dataprep by Trifacta https://cloud.google.com/dataprep

Google Cloud Dataprep API https://clouddataprep.com/documentation/api

All Trifacta API https://api.trifacta.com/