graciousxclusive / pipedream

Connect APIs, remarkably fast. Free for developers.

Pipedream is an integration platform for developers.

We provide a free, hosted platform that makes it easy to connect apps and develop, execute and maintain event-driven workflows. The platform has over 400 fully integrated applications with managed authentication and support for over 1M npm packages.

Key Features:

• Event Sources - Open source components that emit events from services (GitHub, Slack, Airtable, RSS & more).
• Workflows - A sequence of linear steps - just Node.js code - triggered by an event (via event source, HTTP, timer, and more)
• Actions - Prebuilt code steps that you can use in a workflow to perform common operations across Pipedream's 400+ API integrations, for example: sending email, adding a row to a Google Sheet, and more.
• Destinations - Deliver events asynchronously to common destinations like Amazon S3, Snowflake, HTTP and email
• Serverless - No server or cloud resources to manage
• Free - No fees for individual developers (see limits)

Product Demo: YouTube (5 minutes)

You can also get support, raise a bug or feature request, or file a security disclosure.

Workflows

Workflows are a sequence of linear steps - just Node.js code - triggered by an event (via event source, HTTP endpoint, timer, and more). Workflows make it easy to transform data and integrate with 300+ APIs from various apps and services.

• Trigger your workflow on any event (e.g. HTTP requests, on a schedule, and more).
• Add steps to run Node.js code (using virtually any npm package) and prebuilt actions.
• Steps are executed in the order they appear in your workflow.
• Data is shared between steps via step exports.

See our workflow quickstart for a detailed walkthrough of building a workflow end-to-end.

As you build more advanced workflows, you may also find these docs helpful:

• What are events? - events trigger workflow executions
• What are steps? - building blocks you use to create workflows
• Managing workflow state - how to store state in one execution of a workflow that you can read in subsequent executions
• Passing data to steps - steps are just Node functions, and can accept input via step parameters.
• Connected Accounts - how to authenticate to APIs within code steps.
• Error Handling - how to use the Global Error workflow to manage errors raised by workflows.

Event Sources

Pipedream receives data via event sources. Event sources are open source, run on Pipedream's infrastructure and collect data from your own application and/or services like GitHub, DropBox, Zoom, RSS feeds, and more.

Event sources emit new events produced by the service, which can trigger Pipedream workflows, or which you can consume using Pipedream's REST API or a private, real-time SSE stream.

Here is the simplest event source, an HTTP source:

module.exports = {
  name: "http",
  version: "0.0.1",
  props: {
    http: "$.interface.http",
  },
  run(event) {
    console.log(event); // event contains the method, payload, etc.
  },
};

Popular Event Sources:

• Airtable (deploy)
• AWS (deploy)
• Dropbox (deploy)
• GitHub (deploy)
• Google Calendar (deploy)
• Google Drive (deploy)
• RSS (deploy)
• Twitter (deploy)

Event sources can also be deployed via the Pipedream CLI. Once installed, you can deploy an event source by running:

pd deploy   # prompts you to select a component and pass required options

You can create your own event sources for your own personal use. If you think others would benefit from your source, you can publish them to all Pipedream users. See these docs to get started:

• Quickstart for developing a source
• Component API

Actions

Actions are prebuilt code steps that you can use to perform common operations across Pipedream's 400+ API integrations, for example: sending email, adding a row to a Google Sheet, and more. Pipedream supports thousands of prebuilt actions.

Typically, integrating with these services requires a custom code to manage authentication, error handling, etc. Actions abstract that for you: you just pass the necessary params as input, and the action handles the rest. For example, the HTTP GET Request action accepts the data you want to send and the URL you want to send it to, returning the HTTP response for use in future steps.

You can create your own actions, allowing you to re-use them across workflows in your account. You can also publish actions to the entire Pipedream community, making them available for anyone to use.

Here's the code for an action to make an HTTP GET request:

const axios = require('axios')
const http = require('../../http.app.js')

module.exports = {  
  key: "http-get-request",
  name: "GET Request",
  description: "Make an HTTP `GET` request to any URL. Optionally configure query string parameters, headers and basic auth.",
  type: "action",
  version: "0.0.1",
  props: {
    http,
    url: { propDefinition: [http, "url"] },
    params: { propDefinition: [http, "params"] },
    headers: { propDefinition: [http, "headers"] },
    auth: { propDefinition: [http, "auth"] },
  },
  methods: {},
  async run() {
    const config = {
      url: this.url,
      method: "GET",
      params: this.params,
      headers: this.headers,
    }
    if (this.auth) config.auth = this.http.parseAuth(this.auth)
    return (await axios(config)).data
  },
}

Destinations

Destinations, like actions, abstract the connection, batching, and delivery logic required to send events to services like Amazon S3, or targets like HTTP and email.

For example, sending data to an Amazon S3 bucket is as simple as calling $send.s3():

$send.s3({
  bucket: "your-bucket-here",
  prefix: "your-prefix/",
  payload: event.body,
});

Pipedream supports the following destinations today:

• Amazon S3
• Snowflake
• HTTP
• Email
• SSE

Pricing

Pipedream has a generous free tier. You can run sources and workflows for free within the limits of the free tier. If you hit these limits, you can upgrade to one of our paid tiers.

Limits

The Pipedream platform imposes some runtime limits on sources and workflows. Read more about those in our docs.

Found a Bug? Have a Feature to suggest?

Before adding an issue, please search the existing issues or reach out to our team to see if a similar request already exists.

If an issue exists, please add a reaction or comment on your specific use case.

If an issue doesn't yet exist, please use these templates to create one:

New feature

Report a bug

New trigger / source

New API integration

New action

Security

You can read about our platform security and privacy here.

If you'd like to report a suspected vulnerability or security issue, or have any questions about the security of the product, please contact our security team at [email protected].